-- Logs begin at Wed 2022-09-07 13:54:59 UTC, end at Wed 2022-09-07 15:02:52 UTC. -- Sep 07 14:25:43.464416 dv-cinder-801354-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 14:25:43.471559 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 14:25:43 2022] *** Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: nodename: dv-cinder-801354-8-iscsi Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: machine: x86_64 Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: clock source: unix Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: pcre jit disabled Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: detected number of CPU cores: 4 Sep 07 14:25:43.474070 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: current working directory: / Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: detected binary path: /usr/bin/uwsgi-core Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: your processes number limit is 47680 Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: your memory page size is 4096 bytes Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: detected max file descriptor number: 2048 Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: lock engine: pthread robust mutexes Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: thunder lock: enabled Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 14:25:43.474665 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 14:25:43.514589 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: Python main interpreter initialized at 0x5580f11cb9c0 Sep 07 14:25:43.514589 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: python threads support enabled Sep 07 14:25:43.514589 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: your server socket listen backlog is limited to 100 connections Sep 07 14:25:43.514589 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: your mercy for graceful operations on workers is 90 seconds Sep 07 14:25:43.514959 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 14:25:43.515035 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: *** Operational MODE: preforking *** Sep 07 14:25:43.515312 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: *** uWSGI is running in multiple interpreter mode *** Sep 07 14:25:43.515312 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: spawned uWSGI master process (pid: 81790) Sep 07 14:25:43.515517 dv-cinder-801354-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 14:25:43.517624 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: spawned uWSGI worker 1 (pid: 81804, cores: 1) Sep 07 14:25:43.517624 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: spawned uWSGI worker 2 (pid: 81805, cores: 1) Sep 07 14:25:43.518880 dv-cinder-801354-8-iscsi devstack@keystone.service[81790]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 14:25:47.864690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 14:25:47.864690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: from cryptography.utils import int_from_bytes Sep 07 14:25:47.865455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 14:25:47.865455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: from cryptography.utils import int_from_bytes Sep 07 14:25:47.869378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 14:25:47.869378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: from cryptography.utils import int_from_bytes Sep 07 14:25:47.869840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 14:25:47.869840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: from cryptography.utils import int_from_bytes Sep 07 14:25:47.918970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 14:25:47.919164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 14:25:47.919335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] command line args: [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 14:25:47.919609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 14:25:47.919787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ================================================================================ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 14:25:47.920045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] admin_token = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.920325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config_dir = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.920597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.920695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 14:25:47.920767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config_source = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.920895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 14:25:47.920972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] debug = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.921088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] command line args: [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 14:25:47.921174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.921358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 14:25:47.921484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.921568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ================================================================================ {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 14:25:47.921634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.921783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] admin_token = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.921867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] config_dir = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] config_source = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_config_append = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] debug = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.922869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_dir = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.923994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_config_append = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_dir = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_file = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.924909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.925959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.926999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] publish_errors = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.927863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.928004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.928212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.928293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] standard_threads = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] publish_errors = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.932957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_journal = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_json = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_stderr = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_syslog = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] standard_threads = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.933904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.933974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.934043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.934182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] use_journal = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.934402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] use_json = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.934547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] use_stderr = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] use_syslog = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.934829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 14:25:47.934900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.934974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.935967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.936958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.937941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.938971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.939988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.940937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.941538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.941638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.941738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.941840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.941944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.942996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.943997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.944913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.945948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.946203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.946444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.946533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.946688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.946805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.946925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.947996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.948982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.949964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.950937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.951895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.952891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.953870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.954973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.955992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.956227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.956343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.956420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.956498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.956722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.956802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.957491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.957586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.957739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.957817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.957976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.external = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.external = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958708 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.958960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.password = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.token = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.password = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] auth.token = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.959922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.960982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.961948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.962933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.963898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.964993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.965939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.966902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.967885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.968055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.968126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.968763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.968850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.968850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.969957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.970981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.971865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.972753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.978990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979567 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.979996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980068 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.980960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.981957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.982937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.983956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.984979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.985950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.986941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.987992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.988999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.989836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.990971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.991927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] role.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] role.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.992941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.993924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.994926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.995819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.996890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.997880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.998980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:47.999949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.000034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.000213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.000294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.000451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.000527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.008543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.008660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.008660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.008893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.008967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.009973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.010959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.connection = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.011887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.012968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 14:25:48.013807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 14:25:48.013807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81804) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 14:25:48.038938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.039167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.039395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.039778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.040015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.040339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.047816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.048774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.076742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.076968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.077972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.078207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.135169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.137970 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.194814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.195042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.195567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.195802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.362910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.363060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.363142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.363276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.363359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.386952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.387714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.399906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.402274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.413802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.414987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.415106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.415187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.415320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.415532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.415746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81804) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 14:25:48.437824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.438036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 14:25:48.438229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.438385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.438468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 14:25:48.438592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 14:25:48.438788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.438875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.438986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 14:25:48.439100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.439257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.439348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.439514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.439595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.439769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.440024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.449714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.449996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.450176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.450260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.450454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.450537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.450723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.451005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.458048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.458671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.460967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.461260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.461549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.461631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.461836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.461917 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.462915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.463996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.464290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.486347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.486758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.487845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.488225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.488312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.488584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.493038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.493213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.493328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.493498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.493614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.493779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.500060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.500204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.500372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.500458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.506572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.506676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.506866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.506951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.507988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.508079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.508281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.508365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.508552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.508633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.524744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.524847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.525025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.525108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.529070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.529253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.532020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.532121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.532346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.532441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.536525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.536682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.539577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.539783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.542967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.543066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.543267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.543354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.543550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.543900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.543979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.544062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.544165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.544245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.544382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.544483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.559825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.560114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.560218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.560406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.560505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.560728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.560825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.560957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.568179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.568329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.569724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.569973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.571212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.571488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.571640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.571782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.574229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.574468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.574553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.574755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.574839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.575057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.575142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.575437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.575558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.575986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.587103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.587381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.587661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.587829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.587940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.588117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.588345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.588430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.588625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.588706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.588911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.589064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.589364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.589648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.604727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.609034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.609126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.609203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.610550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.611362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.612120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.613424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.614151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.614435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.614827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.615023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.615314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.615717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.621240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.622150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.622694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.623623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.625555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.625948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.626232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.626510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.626607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.626969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.627199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.627480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.634873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.637723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.637723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.637867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.637867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.637867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.642557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81804) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.642835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.643115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.643395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.643489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 14:25:48.643689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.643785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.643961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.644045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.644891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.644983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645416 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.645978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81804) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.646972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.647238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.647499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 14:25:48.680527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5580f11cb9c0 pid: 81804 (default app) Sep 07 14:25:48.681229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5580f11cb9c0 pid: 81805 (default app) Sep 07 14:25:48.683822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102f3a3-15c1-429a-ad64-6868730f8772 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:48.684110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102f3a3-15c1-429a-ad64-6868730f8772 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:48.684402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102f3a3-15c1-429a-ad64-6868730f8772 None None] PATH_INFO: `/v3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:48.688560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Wed Sep 7 14:25:48 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:25:56.463271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:56.463810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:56.463810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:56.468173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:25:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:25:56.473481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:56.473896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:56.474023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:56.546296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: INFO dbcounter [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] Registered counter for database keystone Sep 07 14:25:56.566226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG oslo_db.sqlalchemy.engines [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81804) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 14:25:56.572588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writer thread running {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 14:25:56.750136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG passlib.handlers.bcrypt [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81804) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 14:25:56.750432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG passlib.handlers.bcrypt [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81804) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 14:25:56.808046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:25:56.809656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:25:56.937364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:25:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 14:25:57.417976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:25:57.491195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: INFO dbcounter [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] Registered counter for database keystone Sep 07 14:25:57.512818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG oslo_db.sqlalchemy.engines [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81805) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 14:25:57.520240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writer thread running {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 14:25:57.637154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e473fd-e22a-4367-b885-5d265b809bc4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:25:57.641411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:25:57.642310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:57.642592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:57.642865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:57.651577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:25:57.702586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 14:25:57.702878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d541f96-e245-4605-becf-d3b3bc17a15d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:25:57.712732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Wed Sep 7 14:25:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:25:57.718693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:25:57.733071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6272e-de3d-4f86-a345-6699fa945cb0 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:25:57.737506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:25:57.738598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:57.738875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:57.739142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] PATH_INFO: `/v3/projects/0c46f28c367444249b347e64385a501e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:57.740743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] RBAC: Authorizing `identity:get_project(project_id=0c46f28c367444249b347e64385a501e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:25:57.797084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING keystone.common.rbac_enforcer.enforcer [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 14:25:57.797429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2c6b-0fb7-4812-91f6-fa5622f79c19 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:25:57.802628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Wed Sep 7 14:25:57 2022] GET /identity/v3/projects/0c46f28c367444249b347e64385a501e => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:25:59.916465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b09fe-b95e-4244-95b5-40c7dc518118 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:59.916981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b09fe-b95e-4244-95b5-40c7dc518118 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:59.917063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b09fe-b95e-4244-95b5-40c7dc518118 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:59.921650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:25:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:25:59.927939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:25:59.928337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:25:59.928524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:25:59.948472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:25:59.949983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:25:59.957731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:25:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 14:26:00.230259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b09fe-b95e-4244-95b5-40c7dc518118 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:00.243047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b09fe-b95e-4244-95b5-40c7dc518118 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:00.247383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb3c66-5c20-42b5-abc9-a16cfc0849a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:00.261531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb3c66-5c20-42b5-abc9-a16cfc0849a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:00.261531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb3c66-5c20-42b5-abc9-a16cfc0849a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:00.261531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb3c66-5c20-42b5-abc9-a16cfc0849a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:00.261531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb3c66-5c20-42b5-abc9-a16cfc0849a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:00.261531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb3c66-5c20-42b5-abc9-a16cfc0849a9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:00.267958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Wed Sep 7 14:26:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:26:00.273365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:00.282542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdea23-3d77-4bf7-9f66-52b4d2f7a336 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:00.286852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0b238-fba3-4677-8ceb-597894137f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:00.287541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b238-fba3-4677-8ceb-597894137f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:00.293296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b238-fba3-4677-8ceb-597894137f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:00.293296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b238-fba3-4677-8ceb-597894137f63 admin admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:00.293296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b238-fba3-4677-8ceb-597894137f63 admin admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:00.293296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b238-fba3-4677-8ceb-597894137f63 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:00.293296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Wed Sep 7 14:26:00 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:02.524200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527051c0-e6f7-48c4-a459-fe3211709373 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:02.524813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527051c0-e6f7-48c4-a459-fe3211709373 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:02.524813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527051c0-e6f7-48c4-a459-fe3211709373 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:02.526092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:02.531214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:02.531621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:02.531903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:02.547797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bf45f-e022-4a90-9cb8-a09a073e4139 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:02.548082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bf45f-e022-4a90-9cb8-a09a073e4139 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:02.548390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bf45f-e022-4a90-9cb8-a09a073e4139 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:02.549734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:02.550960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:02.552172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:02.554692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:02.554980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:02.555259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:02.597153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG passlib.handlers.bcrypt [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81805) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 14:26:02.597555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG passlib.handlers.bcrypt [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81805) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 14:26:02.631264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:02.646659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:02.648281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:02.746699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:02.902387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:02.927211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.004377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c45ff1-3c24-46e7-bf83-f637b7243cba None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.024597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.041930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.042575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.042851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.043122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.054852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:03.063633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:03.064741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:03.066229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:03.070883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.124384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333818c-ff99-4841-89cf-02fee1557d8e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.126658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.155779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.165379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.166242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.166520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.166788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.173966 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42438f50-dae4-43b0-95b9-db0c4724be72 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.208498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.209104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.209381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.209651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.210313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:03.210613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.212002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.223752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:03.229205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.234452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.235996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.327623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a6b75-648c-4aac-ab2c-7418f79fa28b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.333808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:26:03.339215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.341857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.362193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.362816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.363091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.363360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.364542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.366062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.367609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:03.372889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.388085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847d181-dfc9-4c97-9301-508f3512fa14 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.472359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.473206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.473481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.473752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.477220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084d4a4-ecd3-48b0-8eeb-97711a5d967c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.491518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.492992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:26:03.511043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:03.511937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:26:03.512779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:26:03.512779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:26:03.512779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:03.512946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.512946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.513100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:26:03.513166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.513166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.513802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.515261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.517086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:03.518460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.522642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:03.576624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.604414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:03.622144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133633d1-6cd3-4de9-b97a-fdf123c85e90 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.623521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0b1b-cb5d-4729-bbec-baf618469392 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:03.654976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.655584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.655866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.656160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.656353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:03.656809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:03.656952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:03.657110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.657233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:03.657504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:03.658218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.domain.id=default, user.id=f0b504610bde4460815c46c6c6298f09) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:03.658490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db281a19-2862-4dd2-b354-31daa97ecea9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.658578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:03.660462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166f20f-a159-43a6-8f3c-82de6020f2bf None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:03.667007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/role_assignments?user.id=f0b504610bde4460815c46c6c6298f09&scope.domain.id=default&role.id=c63cf2d08b334ece887cbd2702835968 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:03.677314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:06.199669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2ed5-0ab4-4711-b2f0-bd22212dc5e9 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.200224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2ed5-0ab4-4711-b2f0-bd22212dc5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.200224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2ed5-0ab4-4711-b2f0-bd22212dc5e9 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.201567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:06.207249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004978-0772-41e7-b34e-5ae4c8951194 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.207686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004978-0772-41e7-b34e-5ae4c8951194 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.207868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004978-0772-41e7-b34e-5ae4c8951194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.233304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-48004978-0772-41e7-b34e-5ae4c8951194 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:06.234731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-48004978-0772-41e7-b34e-5ae4c8951194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:06.326241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 11/23] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:06.550541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7b87-dbde-4cf2-aa5b-f0469017a678 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.550818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7b87-dbde-4cf2-aa5b-f0469017a678 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.551268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7b87-dbde-4cf2-aa5b-f0469017a678 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.552982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:06.558512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.559044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.559514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.585134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:06.586643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:06.597686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ee1ca-924b-4bef-bb98-ef1a828b27fa None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.598247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ee1ca-924b-4bef-bb98-ef1a828b27fa None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.598715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ee1ca-924b-4bef-bb98-ef1a828b27fa None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.600553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 14/25] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:06.605163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e7072-c83d-444a-a0db-d46fdd4eaeb8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.605692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e7072-c83d-444a-a0db-d46fdd4eaeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.606166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e7072-c83d-444a-a0db-d46fdd4eaeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.630565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ed9e7072-c83d-444a-a0db-d46fdd4eaeb8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:06.635429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ed9e7072-c83d-444a-a0db-d46fdd4eaeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:06.675317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:06.710742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 12/26] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:06.745285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 15/27] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:06.802165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:06.890050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3aa98-2696-429e-8494-73d3188655ea None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.890380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3aa98-2696-429e-8494-73d3188655ea None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.890613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3aa98-2696-429e-8494-73d3188655ea None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.892374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 16/28] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:06.898282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.898567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.898843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.927041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a200211-2d79-4c51-91bf-2cb85d8c2f22 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:06.929343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:06.930537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:06.977980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:06.978928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:06.979428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:06.979902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:06.992065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:06.998989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:06.999820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:07.000694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:07.002962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 13/29] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:07.029710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:07.035065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 17/30] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 14:26:07.092688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:07.166396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:07.179192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431953-be2d-4062-9f65-4c106743eab7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:07.188518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:07.243502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:07.244192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.244425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.244705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:07.245386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:07.245699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:07.247304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:07.268942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 14/31] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:07.269759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:07.364127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133aaa-c18f-4f7d-93ec-360682880852 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:07.403119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:07.404068 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.404522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.404941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:07.405799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:07.407349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:07.453997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:07.458793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc8d75-3d89-47e0-ab78-6857dce5f33d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:07.493551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:07.494424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.494853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.495288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:07.496104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:07.497674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:07.635220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 18/32] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Sep 7 14:26:07 2022] POST /identity/v3/roles => generated 225 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:26:07.636668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:07.735332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 15/33] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Sep 7 14:26:07 2022] POST /identity/v3/roles => generated 219 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:07.736776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:07.738040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:07.797118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee836d9-3111-4555-889d-d31730882a33 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:07.853544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b8497-cbd1-4de7-bde3-ca128e0a1049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:07.855302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b8497-cbd1-4de7-bde3-ca128e0a1049 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.855302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b8497-cbd1-4de7-bde3-ca128e0a1049 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.855302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b8497-cbd1-4de7-bde3-ca128e0a1049 None admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:07.856180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b8497-cbd1-4de7-bde3-ca128e0a1049 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:07.857504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b8497-cbd1-4de7-bde3-ca128e0a1049 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:07.859241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 19/34] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:07 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:07.860938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bb920-580d-4332-bbca-06e8ed067559 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.860938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bb920-580d-4332-bbca-06e8ed067559 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.861128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bb920-580d-4332-bbca-06e8ed067559 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:07.862296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 20/35] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:07.867094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bb920-580d-4332-bbca-06e8ed067559 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:07.902542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153ec1-8082-4bed-906d-2cc398136172 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:07.948791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:07.949789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.950265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.950743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:07.962262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:07.963855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-05fca168-75dc-4cf1-9597-391ecc62c18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:26:07.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:07.987629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:26:07.988454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:26:07.988454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:26:07.988454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:07.988454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 16/36] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:07.990273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:07.990564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:07.990845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.006983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bb920-580d-4332-bbca-06e8ed067559 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:08.017881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:08.019805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:08.025421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:08.043939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:08.048458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:08.048992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.049343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.049605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.050845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:08.052445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:08.054326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 21/37] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:08.054978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:08.108183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 17/38] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:08.108183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:08.197954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97648a-d36f-497b-91a0-7884266d2828 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:08.221073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda8e60-21b8-4b9f-bf68-d4491feee36b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:08.233450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:08.234060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.234354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.234607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.235256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:08.235560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:08.237210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751b3f-091f-413e-a1e7-539ced828d29 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:08.254989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 22/39] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:08.256295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5ead-86a8-45ef-8b09-d1e0c6059cce None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.256584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5ead-86a8-45ef-8b09-d1e0c6059cce None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.256856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5ead-86a8-45ef-8b09-d1e0c6059cce None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.260183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 23/40] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:08.260526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5ead-86a8-45ef-8b09-d1e0c6059cce None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:08.262685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111759ba-e6ae-47bb-9494-0964f637692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:08.263498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111759ba-e6ae-47bb-9494-0964f637692f None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.263782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111759ba-e6ae-47bb-9494-0964f637692f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.264101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111759ba-e6ae-47bb-9494-0964f637692f None admin] PATH_INFO: `/v3/domains/default/users/f0b504610bde4460815c46c6c6298f09/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.266818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111759ba-e6ae-47bb-9494-0964f637692f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f0b504610bde4460815c46c6c6298f09, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:08.269006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111759ba-e6ae-47bb-9494-0964f637692f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:08.323824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:08.363483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:08.364730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5ead-86a8-45ef-8b09-d1e0c6059cce None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:08.376178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 18/41] 10.222.0.57 () {58 vars in 1361 bytes} [Wed Sep 7 14:26:08 2022] PUT /identity/v3/domains/default/users/f0b504610bde4460815c46c6c6298f09/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:08.376178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.376178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.376178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.397687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:08.399785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:08.409537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:08.410174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.410459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.410733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.411413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:08.412957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:08.476000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 19/42] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:08.476847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:08.547648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 24/43] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:26:08 2022] POST /identity/v3/projects => generated 301 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:08.599244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05d55-5bdd-4f3a-bbcf-34651ffed238 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:08.634740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:08.642003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:08.642754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:08.643012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:08.643280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:08.644202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:08.651182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:08.733362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 20/44] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Sep 7 14:26:08 2022] POST /identity/v3/roles => generated 223 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:08.749378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:08.866643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fda04-aad8-44aa-826b-2929af26739d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:08.988391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:09.009674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5ae2b-baaa-4f0a-94dd-fc314628d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:09.010315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5ae2b-baaa-4f0a-94dd-fc314628d134 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:09.010597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5ae2b-baaa-4f0a-94dd-fc314628d134 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:09.010940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5ae2b-baaa-4f0a-94dd-fc314628d134 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:09.012006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5ae2b-baaa-4f0a-94dd-fc314628d134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:09.013503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5ae2b-baaa-4f0a-94dd-fc314628d134 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:09.015078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 25/45] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:09.021675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:09.120172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dd558-8d8b-41d1-b052-a8d2e3ccabf5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:09.153624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556faa79-01ed-4233-a36d-a4fa14e1dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:09.154248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556faa79-01ed-4233-a36d-a4fa14e1dddf None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:09.154531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556faa79-01ed-4233-a36d-a4fa14e1dddf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:09.154805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556faa79-01ed-4233-a36d-a4fa14e1dddf None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:09.155472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556faa79-01ed-4233-a36d-a4fa14e1dddf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:09.156963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556faa79-01ed-4233-a36d-a4fa14e1dddf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:09.259882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 21/46] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:26:09 2022] POST /identity/v3/projects => generated 312 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:10.628982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f97821-3655-492b-853c-d2ea36616778 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:10.629522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f97821-3655-492b-853c-d2ea36616778 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:10.629522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f97821-3655-492b-853c-d2ea36616778 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:10.630886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 26/47] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:10.636190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:10.636604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:10.636858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:10.654820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:10.655981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:10.690401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:10.721861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 22/48] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:10.987306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f97821-3655-492b-853c-d2ea36616778 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:11.086713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f97821-3655-492b-853c-d2ea36616778 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:11.120740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:11.121370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.121646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.122011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.136692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:11.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:11.144667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:11.146306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 27/49] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:11.151034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:11.355974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddf3d8-acbb-44c6-ab73-04bbf0f20347 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:11.390616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:11.391226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.391757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.392024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.392735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:11.393035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:11.394403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520d885-9f89-4b0a-b78a-57a38dd9935c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:11.402256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 23/50] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:11.408471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:11.449770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:11.454874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c592c-f60b-4fe5-abdc-1e2b25eb0cfa None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.455191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c592c-f60b-4fe5-abdc-1e2b25eb0cfa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.455474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c592c-f60b-4fe5-abdc-1e2b25eb0cfa None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.456895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 24/51] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:11.462177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.462444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.462710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.480593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:11.481769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:11.510075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f8bb2-5b03-4654-a63a-c165dae445ab None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:11.515560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:11.543981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:11.544642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.544928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.545199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.546331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:11.547809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:11.549922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 28/52] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:11 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:11.555270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:11.555905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 25/53] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:11.652367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f118ab-4947-4bad-b501-91936fc8f0da None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:11.661344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:11.685646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:11.686463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.686998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.687539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.688726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:11.690694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:11.692299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 29/54] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:11.698052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:11.794049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585faab2-41bd-40c5-9843-da18e8d9f5cb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:11.820163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:11.827066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:11.827687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.827968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.828271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.829005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.domain.id=default, user.id=f0b504610bde4460815c46c6c6298f09) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:11.829319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:11.831341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:11.839201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 26/55] 10.222.0.57 () {56 vars in 1322 bytes} [Wed Sep 7 14:26:11 2022] GET /identity/v3/role_assignments?user.id=f0b504610bde4460815c46c6c6298f09&scope.domain.id=default&role.id=c63cf2d08b334ece887cbd2702835968 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:11.928207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77feac-1deb-4da6-a9c4-fe807080ecf7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:11.964555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71cb6c-2303-4f87-9e9d-c42a1e63c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:11.965216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71cb6c-2303-4f87-9e9d-c42a1e63c94f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:11.965502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71cb6c-2303-4f87-9e9d-c42a1e63c94f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:11.965781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71cb6c-2303-4f87-9e9d-c42a1e63c94f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:11.968194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a71cb6c-2303-4f87-9e9d-c42a1e63c94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:11.968833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a71cb6c-2303-4f87-9e9d-c42a1e63c94f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:11.970086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 30/56] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:11.976364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:12.006102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:12.078842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5f659-d5ec-4e4e-b311-b16f22b5d988 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:12.111864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372cbc-f2b5-43ff-8ddc-d68aea3e4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:12.112501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372cbc-f2b5-43ff-8ddc-d68aea3e4b9b None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:12.112784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372cbc-f2b5-43ff-8ddc-d68aea3e4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:12.113057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372cbc-f2b5-43ff-8ddc-d68aea3e4b9b None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:12.113704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3372cbc-f2b5-43ff-8ddc-d68aea3e4b9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:12.115156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3372cbc-f2b5-43ff-8ddc-d68aea3e4b9b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:12.198671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 27/57] 10.222.0.57 () {60 vars in 1140 bytes} [Wed Sep 7 14:26:11 2022] POST /identity/v3/projects => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:14.328411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83455bd-9e3c-4e8c-9013-c38ea0bf8cd8 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:14.328944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83455bd-9e3c-4e8c-9013-c38ea0bf8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:14.329033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83455bd-9e3c-4e8c-9013-c38ea0bf8cd8 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:14.330923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 31/58] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:14.336228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:14.336638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:14.336782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:14.355362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:14.356608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:14.420986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 28/59] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:14.696724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83455bd-9e3c-4e8c-9013-c38ea0bf8cd8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:14.742992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:14.806261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83455bd-9e3c-4e8c-9013-c38ea0bf8cd8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:14.842170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7dbec-e576-41a6-a9f7-24a98e87b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:14.842796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7dbec-e576-41a6-a9f7-24a98e87b333 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:14.843076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7dbec-e576-41a6-a9f7-24a98e87b333 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:14.843351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7dbec-e576-41a6-a9f7-24a98e87b333 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:14.844500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7dbec-e576-41a6-a9f7-24a98e87b333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:14.845996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7dbec-e576-41a6-a9f7-24a98e87b333 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:14.847569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 32/60] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:14.853105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:14.864360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:14.948602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fd946-4815-4585-9891-db777af4b05c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:14.981047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94de5f7-a241-4dc4-959b-7928a1fd61d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:14.981653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94de5f7-a241-4dc4-959b-7928a1fd61d1 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:14.981928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94de5f7-a241-4dc4-959b-7928a1fd61d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:14.982194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94de5f7-a241-4dc4-959b-7928a1fd61d1 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:14.982845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94de5f7-a241-4dc4-959b-7928a1fd61d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:14.984326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94de5f7-a241-4dc4-959b-7928a1fd61d1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:15.091129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 29/61] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:14 2022] POST /identity/v3/users => generated 275 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:26:18.077200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22efd1-7e02-45b0-9d4f-07de669b6b42 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.077700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22efd1-7e02-45b0-9d4f-07de669b6b42 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.077774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22efd1-7e02-45b0-9d4f-07de669b6b42 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.079130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 33/62] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:18.087508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f75e-22f8-465c-8e81-7503baf4003e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.087913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f75e-22f8-465c-8e81-7503baf4003e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.088080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f75e-22f8-465c-8e81-7503baf4003e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.113217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2169f75e-22f8-465c-8e81-7503baf4003e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:18.114433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2169f75e-22f8-465c-8e81-7503baf4003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:18.195595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:18.203106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 30/63] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:18.488969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881dec7-8cab-4ae3-a16e-59c743e979bb None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.489640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881dec7-8cab-4ae3-a16e-59c743e979bb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.490141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881dec7-8cab-4ae3-a16e-59c743e979bb None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.491704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 34/64] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:18.497104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191bff-6f3d-47bd-b429-d764e1357206 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.497603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191bff-6f3d-47bd-b429-d764e1357206 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.498096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191bff-6f3d-47bd-b429-d764e1357206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.520417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e9191bff-6f3d-47bd-b429-d764e1357206 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:18.521928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e9191bff-6f3d-47bd-b429-d764e1357206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:18.607434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 31/65] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:18.630864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881dec7-8cab-4ae3-a16e-59c743e979bb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:18.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881dec7-8cab-4ae3-a16e-59c743e979bb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:18.783483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:18.790947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d42cf4-4707-4f24-88cb-b1d0cbc693f6 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.791610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d42cf4-4707-4f24-88cb-b1d0cbc693f6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.792084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d42cf4-4707-4f24-88cb-b1d0cbc693f6 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.793880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 32/66] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:18.801272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbaa14-9eff-471f-b516-536ae71408ed None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.801897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbaa14-9eff-471f-b516-536ae71408ed None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.802541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbaa14-9eff-471f-b516-536ae71408ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.808092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:18.808775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.809050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.809318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.824620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:18.827659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dfdbaa14-9eff-471f-b516-536ae71408ed None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:18.829119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dfdbaa14-9eff-471f-b516-536ae71408ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:18.829541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:18.830418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:18.831167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:18.835621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 35/67] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:18.836880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:18.944295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 33/68] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:18.945798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011410-7f79-4ce9-b890-d0084f96eb97 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.946085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011410-7f79-4ce9-b890-d0084f96eb97 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.946355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e011410-7f79-4ce9-b890-d0084f96eb97 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.947739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 34/69] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:18.953571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:18.953858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:18.954135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:18.981999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b3513-de01-4d5c-8ca5-dbbc89dd4a8d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:18.984876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:18.986837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:19.018488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.019192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.019415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.019692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.020386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:19.020694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.022304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.036038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 36/70] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:19.036038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.051852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.070340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.080165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 35/71] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:19.080165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.172473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7da8-6186-4bb3-b83e-f527a0857520 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.225741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43abd9-1f2f-4fef-a4df-8c8e719349f4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.229828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.230099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.230481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.230853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.242130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:19.247609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:19.248487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:19.249275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 37/72] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:19.249878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.269911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.270797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.271342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.271750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.293247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.294980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.301673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 36/73] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:19.302789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.366294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.367936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.377941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1e34-bc0d-4778-a3aa-485466a7014e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.414158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.414805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.415086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.415357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.426462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:19.428675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd28e-dbb4-4be8-9d27-e04e35311a1d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:19.431317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:19.432137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:19.432930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 38/74] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:19.437187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.464691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.465305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.465582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.465862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.466518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:19.466820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.468206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.476492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 37/75] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:19.477857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.537482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10d830-48a8-4da7-aeaf-1cb46f993c83 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.570273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0bd2c6-55b3-470b-b632-96b610cac132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.570878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bd2c6-55b3-470b-b632-96b610cac132 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.571165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bd2c6-55b3-470b-b632-96b610cac132 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.571444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bd2c6-55b3-470b-b632-96b610cac132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.574649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdeb0c-932e-493e-94db-6d281fc656c0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.581903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.596302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.609576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.610208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.610489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.610765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.620968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:19.625823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:19.627220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 38/76] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:19.627969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.645190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0bd2c6-55b3-470b-b632-96b610cac132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.647206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0bd2c6-55b3-470b-b632-96b610cac132 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.727857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820345cf-ecef-4076-9cb2-913aedc0c07d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.745707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 39/77] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:19.747065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a519f-e625-4edb-86b2-fa744338f65a None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.747465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a519f-e625-4edb-86b2-fa744338f65a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.747840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a519f-e625-4edb-86b2-fa744338f65a None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.749162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 40/78] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:19.749912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a519f-e625-4edb-86b2-fa744338f65a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.766091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.766956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.767267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.767545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.768230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:19.768532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.769935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.777867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 39/79] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:19.778640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.805800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.808615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:19.852887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a519f-e625-4edb-86b2-fa744338f65a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.875395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbbab6-5890-45a3-848f-c29c63a1fc3e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:19.886980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.887607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.887892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.888184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.889317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.890779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.892673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 41/80] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:19.893436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:19.915327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:19.915938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.916252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.916528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.917189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:19.917496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:19.918872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567eeb-d69a-4489-9cc7-80ef4474f910 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:19.927418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 40/81] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:19.928816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:19.929105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:19.929383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:19.946275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:19.947470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:19.998278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25149ad5-a591-44f6-8c1e-ff233ea80b80 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.014207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 41/82] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:20.015057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.029623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.038207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.041226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.047701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.047701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.047701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.047701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.047701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.047701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 42/83] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:19 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:20.048081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.122941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aa273-345f-4f3c-a3ff-71f17468dec7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.149325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e0925-5bdb-4dea-9a1b-dc9a5c0b5c5b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.160783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.161773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.162378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.162720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.163884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.165580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.167433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 42/84] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:20.167984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.186556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.187210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.187456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.187734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.253249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.263681 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.264226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.266667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.276180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac50f-413d-468c-a744-ac62499b50b0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.310177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.310820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.311099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.311643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.313272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.314734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.316358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 43/85] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:20.317119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.369748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 43/86] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:20.370599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.417764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02ae8-3cd4-46ec-9fac-08437bdadedb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.449985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.450587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.450867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.451140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.451857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=c95f9bc4dec644fdb5e6830927457a4a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:20.452191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.453611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.457475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.459675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 44/87] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/role_assignments?user.id=c95f9bc4dec644fdb5e6830927457a4a&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:20.460475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.481725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac78ef-1cd8-4c00-9377-1ba7aa4ca8c4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.483823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.522127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.522762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.523071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.523316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.575121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d7b8c-3d07-4114-88d2-1ed07aa92f0d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.597564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.599086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.613856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.614826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.615302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.615580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.616808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.618312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.620138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 45/88] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:20.625146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.707016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.722581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 44/89] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:20.722581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.722823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.735882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2b4f3-ae7e-40a5-bab6-342903b88c03 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.772603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.773236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.773518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.773792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.824458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df84b7-4617-43c0-80e5-8fa030c69525 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:20.849308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.851233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.860688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:20.861596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:20.862532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:20.863011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:20.865020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:20.867194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:20.869683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 45/90] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:20.874779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.936076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.952544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:20.968246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 46/91] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:20.969466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:20.987401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e82d6-251b-4f57-bceb-308a3863d342 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:21.025357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:21.025968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:21.026253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:21.026531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:21.027184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:21.028692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:21.074488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e02ec0-8cba-4489-872f-fc9773604a5b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:21.092006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 46/92] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:20 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:21.101303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:21.114023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:21.115297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:21.115297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:21.115631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:21.117098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:21.118754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:21.126074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:20 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:21.127163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:21.194204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:21.248480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842141d-33ac-412b-904a-f58615b95116 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:21.264658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71320e-844f-49f5-b9da-77b98e554906 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:21.286127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:21.300262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:21.300262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:21.300262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:21.300262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] PATH_INFO: `/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:21.301256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] RBAC: Authorizing `identity:get_project(project_id=b0d1a7f2ec6745a18dbcb186d022ebc8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:21.302432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:21.304568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 48/94] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:21 2022] GET /identity/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:21.305289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:21.320122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:21.323323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:21.323323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:21.323323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:21.323323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=f0b504610bde4460815c46c6c6298f09) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:21.323323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:21.323945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:21.331019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 47/95] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:21 2022] GET /identity/v3/role_assignments?user.id=f0b504610bde4460815c46c6c6298f09&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:21.332016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:21.445585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07eb91d-ee37-473d-b199-4568f67e4b82 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:21.477867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:21.489319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2585d-9ae1-403f-83b7-b621201f73de None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:21.491843 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:21.492630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:21.493760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:21.493760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:21.494229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] RBAC: Adding query filter params (role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=c95f9bc4dec644fdb5e6830927457a4a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:21.494601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:21.496249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2e3a6-0297-4543-a1db-2b251a80e00b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:21.510636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 49/96] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:21 2022] GET /identity/v3/role_assignments?user.id=c95f9bc4dec644fdb5e6830927457a4a&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:21.538977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:21.540022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:21.540343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:21.540621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:21.541393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=b0d1a7f2ec6745a18dbcb186d022ebc8, user.id=c95f9bc4dec644fdb5e6830927457a4a) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:21.541709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:21.543240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793dfaf-ab92-40e8-9c9b-19f9b162ba7c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:21.550012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 48/97] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:21 2022] GET /identity/v3/role_assignments?user.id=c95f9bc4dec644fdb5e6830927457a4a&scope.project.id=b0d1a7f2ec6745a18dbcb186d022ebc8&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:23.057720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e09779-ccab-45b4-9379-98693b77f194 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:23.058780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e09779-ccab-45b4-9379-98693b77f194 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:23.059250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e09779-ccab-45b4-9379-98693b77f194 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:23.061086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 50/98] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:23.066023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:23.066794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:23.067293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:23.087927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:23.089763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:23.132264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:23.174705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 49/99] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:23.709655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e09779-ccab-45b4-9379-98693b77f194 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:23.830315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e09779-ccab-45b4-9379-98693b77f194 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:23.873239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:23.874226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:23.874697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:23.875156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:23.888076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:23.894395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:23.895228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:23.895996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:23.898642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 51/100] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:23.904124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:24.019875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aa840-4f96-409e-a5a5-03c442df7455 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:24.054961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:24.055880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.056432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.056915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.057766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:24.058259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:24.059911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:24.068859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 50/101] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:24.077208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:24.148426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:24.213942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571e1e9-c365-48ee-9d37-5cf6354bcee0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:24.262090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101df9e0-a333-4fec-bd64-405bb0ef4a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:24.263051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df9e0-a333-4fec-bd64-405bb0ef4a2d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.263510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df9e0-a333-4fec-bd64-405bb0ef4a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.263939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df9e0-a333-4fec-bd64-405bb0ef4a2d None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.265354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101df9e0-a333-4fec-bd64-405bb0ef4a2d None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:24.268081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101df9e0-a333-4fec-bd64-405bb0ef4a2d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:24.269940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 52/102] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:24 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:24.275721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:24.321316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:24.339915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb979f-b3ff-4fc0-afa3-6d736bf9d170 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.340215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb979f-b3ff-4fc0-afa3-6d736bf9d170 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.340488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb979f-b3ff-4fc0-afa3-6d736bf9d170 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.342010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 53/103] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:24.353838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7122d9-1c7f-409c-980e-820a0c613296 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.354116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7122d9-1c7f-409c-980e-820a0c613296 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.354389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7122d9-1c7f-409c-980e-820a0c613296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.383473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2b7122d9-1c7f-409c-980e-820a0c613296 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:24.386983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2b7122d9-1c7f-409c-980e-820a0c613296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:24.428640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e800-6a79-4a42-9a5a-b3fe78a8f1a3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:24.465179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec627f86-830d-4874-ba22-c15d718e5982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:24.465817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec627f86-830d-4874-ba22-c15d718e5982 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.466101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec627f86-830d-4874-ba22-c15d718e5982 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.466377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec627f86-830d-4874-ba22-c15d718e5982 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.467532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec627f86-830d-4874-ba22-c15d718e5982 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:24.469047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec627f86-830d-4874-ba22-c15d718e5982 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:24.470993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 51/104] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:24 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:24.471943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae978c-b2ec-4c1d-b096-45d0ee04fcf2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.472334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae978c-b2ec-4c1d-b096-45d0ee04fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.472523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae978c-b2ec-4c1d-b096-45d0ee04fcf2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.473733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 52/105] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:24.478469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae978c-b2ec-4c1d-b096-45d0ee04fcf2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:24.536826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 54/106] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:24.538104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc5944-ac52-451d-97bf-fd28789f8606 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.538392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc5944-ac52-451d-97bf-fd28789f8606 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.538670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc5944-ac52-451d-97bf-fd28789f8606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.578450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-13cc5944-ac52-451d-97bf-fd28789f8606 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:24.580251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-13cc5944-ac52-451d-97bf-fd28789f8606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:24.624285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae978c-b2ec-4c1d-b096-45d0ee04fcf2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:24.626042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:24.683472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:24.684455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.684919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.685206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/c95f9bc4dec644fdb5e6830927457a4a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.693128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] RBAC: Authorizing `identity:create_grant(project_id=a08919df664745e6bf377a7cdfd3cb8e, user_id=c95f9bc4dec644fdb5e6830927457a4a, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:24.695349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:24.700527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:24.749699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 55/107] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:24.800125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080e82-3ee6-4f9d-90ad-bc5393d9965b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.800430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080e82-3ee6-4f9d-90ad-bc5393d9965b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.800723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080e82-3ee6-4f9d-90ad-bc5393d9965b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.802236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 56/108] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:24.807124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cb99f-0222-44f3-ab54-52d522ba3ce8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:24.807409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cb99f-0222-44f3-ab54-52d522ba3ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:24.807829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cb99f-0222-44f3-ab54-52d522ba3ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:24.818956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 53/109] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:24 2022] PUT /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/c95f9bc4dec644fdb5e6830927457a4a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:24.834561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bf1cb99f-0222-44f3-ab54-52d522ba3ce8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:24.835771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bf1cb99f-0222-44f3-ab54-52d522ba3ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:24.888055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:24.947298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 57/110] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:25.045422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80bb00-2efc-4aeb-bf3c-cdc09ac2ab13 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.097806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.098405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.098405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.098641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.102834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd7038-6608-429b-814c-2598a4fa5712 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.103297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd7038-6608-429b-814c-2598a4fa5712 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.103367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd7038-6608-429b-814c-2598a4fa5712 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.105305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 58/111] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:25.114957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.115251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.115523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.116371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:25.122140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:25.123029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:25.124005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 54/112] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:25.130611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.155949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:25.158178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:25.161232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:25.248378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a0f90-6546-487c-a244-7fda66007bd7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.263848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 59/113] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:25.267804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.291928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.292798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.293146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.293525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.300908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:25.301326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:25.302936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4370f-fb92-4b48-87d2-030cf6675199 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:25.307196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:25.317927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 55/114] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:25.322688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd9cb3-2e02-48b6-b8c6-f8f0037f34b3 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.322688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd9cb3-2e02-48b6-b8c6-f8f0037f34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.322688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd9cb3-2e02-48b6-b8c6-f8f0037f34b3 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.322688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 56/115] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:25.328567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd9cb3-2e02-48b6-b8c6-f8f0037f34b3 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.397932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deb7c5-12f8-4b8b-a11c-0f72be8dafb8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.431546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:25.442533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed19ff-c34f-43af-bebd-b257f0beceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.443536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed19ff-c34f-43af-bebd-b257f0beceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.448228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed19ff-c34f-43af-bebd-b257f0beceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.448228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed19ff-c34f-43af-bebd-b257f0beceb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.451739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd9cb3-2e02-48b6-b8c6-f8f0037f34b3 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.461300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed19ff-c34f-43af-bebd-b257f0beceb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-deed19ff-c34f-43af-bebd-b257f0beceb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:25.469345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:25.470815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:25.471575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 60/116] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:25.473118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.476425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.476975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.498747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.499394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.499710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.499995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.501251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:25.502326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:25.502890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:25.504016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:25.509180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 57/117] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:25.510110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.590392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 61/118] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:25.591693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.622143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:25.646622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7ded-d897-4895-88af-0394a814804f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.698866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.699612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.700534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.702188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:25.703694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:25.706012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 58/119] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:25.706760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.708683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b1e4e-9b85-4dac-b027-a5b1b2deee7c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.735108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:25.756817 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.757888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.758395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.758905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.759814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:25.760339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:25.761918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:25.778091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 62/120] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:25.779466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.823722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff1959-9645-4318-a3a6-b058d750a7e8 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.863567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.864310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.864597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.864873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.866100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:25.868311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:25.869817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 59/121] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:25.872283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.894093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:25.919932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99d002-f653-4a16-9444-e853b175a423 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:25.960763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:25.961342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:25.961626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:25.961899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:25.973977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:25.978940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:25.979780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 63/122] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:25.980880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:25.986430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:26.003905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0878353-9859-40b3-a16d-513d8be8a378 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.046522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.047142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.047423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.047693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.048429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.049916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.098152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b2141-dda2-4b66-8a40-22350b7a9c29 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.131831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.132498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.132779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.133054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.134187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.135679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.137284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 64/123] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:25 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:26.137284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.155714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 60/124] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:26:25 2022] POST /identity/v3/projects => generated 302 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:26.155714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.159304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:26.264188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45939a-0275-4bb2-90ff-86442a8c2ce5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.295683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c56b7-21b2-4ecb-9b98-791a4acfc9ad None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.306826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.307725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.308271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.308648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/c95f9bc4dec644fdb5e6830927457a4a/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.311129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] RBAC: Authorizing `identity:create_grant(project_id=a08919df664745e6bf377a7cdfd3cb8e, user_id=c95f9bc4dec644fdb5e6830927457a4a, role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.312854 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.340794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.341443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.341722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.342000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.352586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:26.355639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:26.361716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:26.362569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:26.363443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 61/125] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:26.364092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.400765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 65/126] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:26 2022] PUT /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/c95f9bc4dec644fdb5e6830927457a4a/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:26.402039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.500921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3271a2-825e-43a4-af7b-6cf10d0615ed None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.531233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:26.565890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.566521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.566800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.567072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.567759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:26.568063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.569523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.579663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d99c35-1c40-4420-97a1-29e1006e976d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.591897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 62/127] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:26.592911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.648510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.649672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.650150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.656990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.659692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.661216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.662982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 66/128] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:26 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:26.669384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.687060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:26.735070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf31b3-ff89-41e1-b531-fd7b3637830f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.789551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.790228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.790514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.790787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.791465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:26.791771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.794566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.812092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 63/129] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:26.812967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.815000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23f7d9-80b2-40e8-962f-98abcccaf9fd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:26.881305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:26.882337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:26.882942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:26.883538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] PATH_INFO: `/v3/users/7f8856894b404eaba9327c3d486ad0f9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:26.917476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] RBAC: Authorizing `identity:get_user(user_id=7f8856894b404eaba9327c3d486ad0f9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:26.918978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:26.922089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 67/130] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:26 2022] GET /identity/v3/users/7f8856894b404eaba9327c3d486ad0f9 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:26.922724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:26.933490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:26.993026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cb16-ac74-48fd-97a9-268c3ea980df None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:27.026351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:27.035811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:27.036605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.036892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.037170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/f0b504610bde4460815c46c6c6298f09/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.049879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] RBAC: Authorizing `identity:create_grant(project_id=a08919df664745e6bf377a7cdfd3cb8e, user_id=f0b504610bde4460815c46c6c6298f09, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:27.049879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:27.127948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d02627-22cd-483f-8f2a-10358e99ac3c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:27.142511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 64/131] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:26 2022] PUT /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/f0b504610bde4460815c46c6c6298f09/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:27.143848 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:27.172775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:27.173994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.174325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.174600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.177089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:27.178767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:27.180595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 68/132] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:26 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:27.187069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:27.284556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:27.364641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47a3b-c41e-4d1e-a33f-2c97f8086044 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:27.398201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55571844-6f6b-4b00-a094-0a59692a19ce None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:27.436058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:27.436731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.437065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.437341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] PATH_INFO: `/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.440294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] RBAC: Authorizing `identity:get_project(project_id=b0d1a7f2ec6745a18dbcb186d022ebc8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:27.441804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:27.443519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 69/133] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:27 2022] GET /identity/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8 => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:27.451273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:27.483813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e015-0bf5-42fd-828b-9ca8e6c263fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:27.484468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e015-0bf5-42fd-828b-9ca8e6c263fa None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.484751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e015-0bf5-42fd-828b-9ca8e6c263fa None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.485022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e015-0bf5-42fd-828b-9ca8e6c263fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.612732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:27.646392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e015-0bf5-42fd-828b-9ca8e6c263fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:27.648767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e015-0bf5-42fd-828b-9ca8e6c263fa None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:27.661895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53da71-acf7-4eb9-b03a-502a2577da07 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:27.729073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81877a2-a2b9-484e-874d-24748a7649f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:27.729745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81877a2-a2b9-484e-874d-24748a7649f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.730020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81877a2-a2b9-484e-874d-24748a7649f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.730291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81877a2-a2b9-484e-874d-24748a7649f9 None admin] PATH_INFO: `/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8/users/c95f9bc4dec644fdb5e6830927457a4a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.733005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81877a2-a2b9-484e-874d-24748a7649f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0d1a7f2ec6745a18dbcb186d022ebc8, user_id=c95f9bc4dec644fdb5e6830927457a4a, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:27.734462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81877a2-a2b9-484e-874d-24748a7649f9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:27.796753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 65/134] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:27.798836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3e23-b2b1-48b8-b558-19e36cb682c1 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.799129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3e23-b2b1-48b8-b558-19e36cb682c1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.799401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3e23-b2b1-48b8-b558-19e36cb682c1 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.800668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 66/135] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:27.803184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3e23-b2b1-48b8-b558-19e36cb682c1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:27.846456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:27.853945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 70/136] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:27 2022] PUT /identity/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8/users/c95f9bc4dec644fdb5e6830927457a4a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:27.856306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:27.856648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:27.856923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:27.896009 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:27.898795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:27.970561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:28.009032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3e23-b2b1-48b8-b558-19e36cb682c1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:28.041814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 71/137] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:28.062007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:28.062752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:28.063107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:28.063451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:28.064840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:28.066485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:28.068696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 67/138] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:27 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:28.077423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:28.221862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551271e7-770f-4dbe-b34c-872b7354e325 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:28.255614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:28.265002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:28.265002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:28.265187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:28.266706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:28.266706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] RBAC: Adding query filter params (role.id=6c4dc8189192426f8d9b637b0f123425, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=7f8856894b404eaba9327c3d486ad0f9) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:28.266706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:28.268103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:28.274902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 72/139] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:28 2022] GET /identity/v3/role_assignments?user.id=7f8856894b404eaba9327c3d486ad0f9&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=6c4dc8189192426f8d9b637b0f123425 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:28.572047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:28.697234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229d2e-c943-4cbb-a56a-568a14d216a9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:28.743006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:28.743648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:28.743924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:28.744296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:28.761012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:28.765878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:28.767570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:28.769029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 68/140] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:28.775646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:28.918057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cde16-df02-4c57-ab5e-bae6138b0798 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:28.961911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:28.962553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:28.962832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:28.963104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:28.963800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:28.966642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:28.966642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:28.988772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 73/141] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:28.994430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:29.020380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:29.125026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb9fd-ae84-4e50-87cb-6361274b6ae1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:29.169833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5b1f6-337d-465f-b868-a6cc743a2965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:29.170400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5b1f6-337d-465f-b868-a6cc743a2965 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.170679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5b1f6-337d-465f-b868-a6cc743a2965 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.170957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5b1f6-337d-465f-b868-a6cc743a2965 None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.172092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5b1f6-337d-465f-b868-a6cc743a2965 None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:29.174161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5b1f6-337d-465f-b868-a6cc743a2965 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:29.176175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 69/142] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:28 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:29.181637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:29.197071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c2a0-7753-4ddd-aef7-9748db62f41a None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.197385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c2a0-7753-4ddd-aef7-9748db62f41a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.197629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c2a0-7753-4ddd-aef7-9748db62f41a None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.198977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 70/143] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:29.213282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e021-8290-4d4c-9497-25750372fc43 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.213573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e021-8290-4d4c-9497-25750372fc43 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.213849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e021-8290-4d4c-9497-25750372fc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.235007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2729e021-8290-4d4c-9497-25750372fc43 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:29.236176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2729e021-8290-4d4c-9497-25750372fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:29.293952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:29.324188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 71/144] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:29.384307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1624ebd-b280-4d3c-90fb-af8b59ab66f0 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.384485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a520da-35cf-4239-94d6-6cff522c60c5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:29.384696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1624ebd-b280-4d3c-90fb-af8b59ab66f0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.384869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1624ebd-b280-4d3c-90fb-af8b59ab66f0 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.386899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 72/145] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:29.405419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.405724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.405978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.442420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:29.443012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.443289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.443561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.445112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:29.446329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:29.453640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:29.561317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:29.562801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:29.626619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 73/146] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:29.647194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:29.715782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 74/147] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:29.721430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:29.805570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:29.856286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf31d17-a7f8-4986-8f6c-dc467b8d7f99 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:29.892126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:29.892800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:29.893074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:29.893342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:29.894480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:29.895942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:29.897812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 74/148] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:29 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:29.903310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:29.914093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:29.989749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebbe28-1105-47f3-a685-805f6b965e5a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.006263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e39d8a-cf0d-48cd-b9c7-430049cd8515 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.025734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.026375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.026651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.026924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.028034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:30.029518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:30.031396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 75/149] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:30.036666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:30.043412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.044027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.044343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.044622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.045369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=c95f9bc4dec644fdb5e6830927457a4a) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:30.045680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:30.047155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:30.053804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 75/150] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:29 2022] GET /identity/v3/role_assignments?user.id=c95f9bc4dec644fdb5e6830927457a4a&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:30.061296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:30.080565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:30.222424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:30.243053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c0108-2f05-4c62-a94d-32de0aad1c71 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.312464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110e1cd-580b-433d-b556-61e20df5e602 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.330857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.332491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.333003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.333474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.334332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:30.336766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:30.383700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.384089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.384305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.384562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.406976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:30.415250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:30.416079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:30.416884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 76/151] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:30.424914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:30.462326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 76/152] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:30 2022] POST /identity/v3/users => generated 283 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:26:30.555458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162698f0-431c-42d5-b224-bfbe663df698 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.588742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:30.594935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.595617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.595948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.596252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.596929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:30.600752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:30.600752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe53a10-933b-4214-a892-68355fa4c4a1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:30.608757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 77/153] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:30.615023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:30.627051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405b99-b5a8-489e-b09c-10d967e2ca3a None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.627051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405b99-b5a8-489e-b09c-10d967e2ca3a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.627051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405b99-b5a8-489e-b09c-10d967e2ca3a None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.628968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 78/154] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:30.636316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.636918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.637393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.657996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:30.659568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:30.740855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 79/155] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:30.751867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:30.775318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838d455-b3eb-47e5-a4c8-6baa8dd2af86 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.812632 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.813216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.813529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.813809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:30.814969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:30.816570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:30.818364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 77/156] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:30 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:30.824324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:30.939667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af000fce-45a0-4ab7-a9d1-4a01a92bae2f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:30.964798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:30.978175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:30.978799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:30.979075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:30.979345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.050143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:31.051654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:31.069797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:31.185418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 80/157] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:31.192410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:31.235809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f80533-80cc-4a6a-86ee-d9feb1b3008a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:31.237346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:31.287638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:31.288840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.288840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.289192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.301309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:31.307093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:31.309061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:31.309886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 78/158] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:31.315853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:31.327616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:31.345371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73c43-54a9-4de5-a53b-e83a6bd87a30 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:31.388698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb6e20-ca13-4c18-88c2-85be7c814887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:31.389340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb6e20-ca13-4c18-88c2-85be7c814887 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.389618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb6e20-ca13-4c18-88c2-85be7c814887 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.389886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb6e20-ca13-4c18-88c2-85be7c814887 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.391381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb6e20-ca13-4c18-88c2-85be7c814887 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:31.393073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb6e20-ca13-4c18-88c2-85be7c814887 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:31.394847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 81/159] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:31 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:31.396240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b19cc-99e6-4c15-b3a4-d958cea1a3d2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.396528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b19cc-99e6-4c15-b3a4-d958cea1a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.396799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b19cc-99e6-4c15-b3a4-d958cea1a3d2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.398019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 82/160] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:31.400896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b19cc-99e6-4c15-b3a4-d958cea1a3d2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:31.429070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce4054-07b3-4a88-86ed-cbc99b7cb2fd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:31.469053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:31.469867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.470148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.471693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.472462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:31.472908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:31.474609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c9e1b-46c6-4379-8fcf-6b1d666be24b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:31.485224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 79/161] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:31.486597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f375b63-7ea8-4403-afc2-adf4f479547b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.486880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f375b63-7ea8-4403-afc2-adf4f479547b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.487154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f375b63-7ea8-4403-afc2-adf4f479547b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.508301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9f375b63-7ea8-4403-afc2-adf4f479547b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:31.509515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9f375b63-7ea8-4403-afc2-adf4f479547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:31.510225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:31.524536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b19cc-99e6-4c15-b3a4-d958cea1a3d2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:31.569347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:31.570193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.570862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.571133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.571867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] RBAC: Adding query filter params (role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=c95f9bc4dec644fdb5e6830927457a4a) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:31.573169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:31.575397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd1999-fbac-42be-8d63-83f20374cd6d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:31.582063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 83/162] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:31 2022] GET /identity/v3/role_assignments?user.id=c95f9bc4dec644fdb5e6830927457a4a&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:31.583306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b6872-fb2a-4687-a260-f41237447f29 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.583588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b6872-fb2a-4687-a260-f41237447f29 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.583857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b6872-fb2a-4687-a260-f41237447f29 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.584303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:31.585509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 84/163] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:31.586079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b6872-fb2a-4687-a260-f41237447f29 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:31.591717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 80/164] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:31.595484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.595767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.596042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.634561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:31.635782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:31.741718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b6872-fb2a-4687-a260-f41237447f29 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:31.800652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:31.801719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.802230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.802701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] PATH_INFO: `/v3/users/f0b504610bde4460815c46c6c6298f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:31.805209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b504610bde4460815c46c6c6298f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:31.806353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:31.807988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 85/165] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:31 2022] GET /identity/v3/users/f0b504610bde4460815c46c6c6298f09 => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:31.814209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:31.821844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 81/166] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:31.847915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:31.933994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43928b9a-0b49-4620-ac73-aa5492dc9060 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:31.977582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:31.978217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:31.978534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:31.978806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:32.060239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:32.061714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:32.090983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:32.180216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 86/167] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:26:32.184410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.186159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.306097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e4ffd-1bfc-401d-9326-bc2691a1adfc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:32.353949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81edb-fe29-40ab-a16b-b817e2e55f5a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:32.368369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:32.386220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:32.387744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:32.388032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:32.388327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:32.400962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:32.409873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:32.414820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:32.415889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:32.416692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 82/168] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:32.425731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.429210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:32.429924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:32.430206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:32.430531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:32.431631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:32.433133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:32.438959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 87/169] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:32.440012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.546478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41310432-1c2e-48d0-ba33-cad24a4590dc None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:32.589403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:32.590974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:32.591678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:32.592858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:32.593930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:32.595989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:32.597639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:32.611211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 88/170] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:32.613438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dff57-d792-422e-bd1a-2b1b1d28b3c4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:32.616447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.682250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:32.683287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:32.683799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:32.684293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:32.706706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:32.711936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:32.712826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:32.713644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:32.717475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 83/171] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:32.718663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.732505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:32.812767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7c16-666c-40b5-88d9-a9845600f401 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:32.818730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:32.878469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:32.879082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:32.879360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:32.879634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:32.880387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=f0b504610bde4460815c46c6c6298f09) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:32.880698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:32.882324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:32.898776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 89/172] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/role_assignments?user.id=f0b504610bde4460815c46c6c6298f09&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:32.899617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:32.918896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29388-b392-43f8-b7ce-b6e013a3cb09 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:32.999269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:32.999973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.000536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.001533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] PATH_INFO: `/v3/users/7f8856894b404eaba9327c3d486ad0f9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.004546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] RBAC: Authorizing `identity:get_user(user_id=7f8856894b404eaba9327c3d486ad0f9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:33.006156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:33.010896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 84/173] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/users/7f8856894b404eaba9327c3d486ad0f9 => generated 289 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:33.017994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:33.123500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5fb5-c5e6-4fa8-b40f-31e8199b0d13 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:33.157892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:33.184450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9d0d-fe30-46d0-a68e-3a3e24c13fcb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:33.185521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:33.186243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.186555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.186898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.187579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:33.188049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:33.189552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:33.202057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:33.208520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 90/174] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:33.224245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:33.226426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:33.227071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.227349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.227734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.228894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:33.230331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:33.232311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 85/175] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:33 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:33.241161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:33.408431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e363-5400-45ec-9426-0ff05e62f346 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:33.417025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f743a-098f-43f4-8e51-7905a63e52d3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:33.463316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b715a-c549-41f4-bf46-dc5581a888ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:33.464001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b715a-c549-41f4-bf46-dc5581a888ec None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.464310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b715a-c549-41f4-bf46-dc5581a888ec None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.464592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b715a-c549-41f4-bf46-dc5581a888ec None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/7f8856894b404eaba9327c3d486ad0f9/roles/6c4dc8189192426f8d9b637b0f123425` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.467153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b715a-c549-41f4-bf46-dc5581a888ec None admin] RBAC: Authorizing `identity:create_grant(project_id=a08919df664745e6bf377a7cdfd3cb8e, user_id=7f8856894b404eaba9327c3d486ad0f9, role_id=6c4dc8189192426f8d9b637b0f123425)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:33.469113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b715a-c549-41f4-bf46-dc5581a888ec None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:33.493549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:33.494129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.494407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.494677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] PATH_INFO: `/v3/users/c95f9bc4dec644fdb5e6830927457a4a` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.496911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] RBAC: Authorizing `identity:get_user(user_id=c95f9bc4dec644fdb5e6830927457a4a)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:33.498647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:33.502411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 91/176] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:33 2022] GET /identity/v3/users/c95f9bc4dec644fdb5e6830927457a4a => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:33.507618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:33.560217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 86/177] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:33 2022] PUT /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/users/7f8856894b404eaba9327c3d486ad0f9/roles/6c4dc8189192426f8d9b637b0f123425 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:33.571751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:33.629928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3448a-e10c-4925-afec-835dade5da63 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:33.668429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:33.669417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.669937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.670399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.750168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:33.752553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:33.797146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e42090-b12a-4387-b11a-576755b6959a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.798230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e42090-b12a-4387-b11a-576755b6959a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.798230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e42090-b12a-4387-b11a-576755b6959a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.799548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 87/178] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:33.804927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d755b8-6748-4562-958d-abecad53d326 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:33.805100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d755b8-6748-4562-958d-abecad53d326 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:33.805428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d755b8-6748-4562-958d-abecad53d326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:33.824557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:33.861600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-25d755b8-6748-4562-958d-abecad53d326 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:33.862811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-25d755b8-6748-4562-958d-abecad53d326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:33.869227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 92/179] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:26:33.875391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:33.932236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:33.974262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 88/180] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:33.999915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d2ef-3fdf-4349-b56f-a8e208101d57 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:34.042160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:34.043196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.043702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.044234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] PATH_INFO: `/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.048952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] RBAC: Authorizing `identity:get_project(project_id=b0d1a7f2ec6745a18dbcb186d022ebc8)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:34.050445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:34.052244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 93/181] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:33 2022] GET /identity/v3/projects/b0d1a7f2ec6745a18dbcb186d022ebc8 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:34.058341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d755b8-6748-4562-958d-abecad53d326 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:34.203610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d755b8-6748-4562-958d-abecad53d326 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:34.247654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:34.248503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.248779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.249052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.249776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=b0d1a7f2ec6745a18dbcb186d022ebc8, user.id=c95f9bc4dec644fdb5e6830927457a4a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:34.250079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:34.252043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18be565-7db3-44ad-bff9-1c1252db3a28 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:34.258110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 89/182] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:34 2022] GET /identity/v3/role_assignments?user.id=c95f9bc4dec644fdb5e6830927457a4a&scope.project.id=b0d1a7f2ec6745a18dbcb186d022ebc8&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:34.330962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:34.350958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:34.436677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce05d43-4eaf-4d56-8d3a-51bc63eec2a1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.437035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce05d43-4eaf-4d56-8d3a-51bc63eec2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.437234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce05d43-4eaf-4d56-8d3a-51bc63eec2a1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.439482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 90/183] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:34.445282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.445576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.445837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.467449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d62c-b5c9-46f0-8de4-8377bd193bfb None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:34.476561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:34.477778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:34.517106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:34.520469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:34.521119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.521401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.521675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.533737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:34.538756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:34.539974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:34.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 94/184] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:34.546102 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:34.568681 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 91/185] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:34.637134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:34.659155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c5545-6f09-410b-8118-64a0d48d8bcf None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:34.695496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:34.696110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.696413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.696686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.697366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:34.697670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:34.699057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ee8c-7fa8-493e-8159-e998ea710fcf None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:34.708004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 95/186] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:34.713768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:34.733664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bbe88-ed9e-48cd-b8be-a082160863c9 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.734248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bbe88-ed9e-48cd-b8be-a082160863c9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bbe88-ed9e-48cd-b8be-a082160863c9 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.737081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 96/187] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:34.741919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.742439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.742899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.769589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:34.771145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:34.847000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 97/188] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:34.853133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440fc80-c851-49a2-9fa7-2da3c5e36ffd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:34.878175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:34.895647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:34.896261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:34.896552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:34.896966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] PATH_INFO: `/v3/users/54c963c9cd174621ad9e153ca2946c62` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:34.915430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] RBAC: Authorizing `identity:get_user(user_id=54c963c9cd174621ad9e153ca2946c62)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:34.917231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:34.919502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 92/189] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:34 2022] GET /identity/v3/users/54c963c9cd174621ad9e153ca2946c62 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:34.926065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:34.955439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:34.970691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.012630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8d2bc-ff79-4061-be6c-55277efe19f6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.046493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eadee-233d-4554-a268-216c870b3c7b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.055107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.055786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.056070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.056673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.067712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:35.072673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:35.073539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:35.074334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 98/190] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:26:35.082108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.093630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.094491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.094788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.095061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.164717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.166114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.191948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496fa7a-4faa-44cb-9b85-416e38eeca32 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.193846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.220095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.227942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.228580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.228861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.229132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.229793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:35.230091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.231470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.239791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 99/191] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:35.244501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.272137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 93/192] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:35.273083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.364199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c04958-91d0-4e25-9eb3-c112bdd4ba2f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.378083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da46f4-2cf8-43e5-abd4-8ae77cc76ab3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.400843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.401499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.401852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.402119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.403335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.404813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.405870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.412226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 100/193] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:35.412226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.418408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.418663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.419041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.419350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] PATH_INFO: `/v3/users/54c963c9cd174621ad9e153ca2946c62` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.420531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] RBAC: Authorizing `identity:get_user(user_id=54c963c9cd174621ad9e153ca2946c62)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.421983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.424108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 94/194] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/users/54c963c9cd174621ad9e153ca2946c62 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:35.424901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.477525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.520921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99254781-2cb2-41cf-b9fb-3380a0c52709 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.529450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e04a1-5e4f-4139-9eb2-7594b2f4ec66 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.560580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.561522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.561872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.562151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.563831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.565323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.566965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 101/195] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:35.567820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.568659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.569049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.569327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.569936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.570771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.572554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.670529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c147bd-d0ea-49a9-baa6-980383feff09 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.682959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 95/196] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:35 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:35.683750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:35.707453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.717717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.718510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.718951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.719855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.766750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:35.805000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.806822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.830871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c5fed-6286-464f-8ec6-e993e4c72bfc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:35.869091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:35.869880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:35.870255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:35.870624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:35.871510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=54c963c9cd174621ad9e153ca2946c62) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:35.871916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:35.873591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:35.880532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 96/197] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/role_assignments?user.id=54c963c9cd174621ad9e153ca2946c62&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:35.939356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 102/198] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:35.945394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:36.067344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f45b8-7730-4886-9bb5-09fcd1c59081 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:36.083574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:36.107205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426704b1-1c0f-4d11-be0e-8798490c425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:36.107860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426704b1-1c0f-4d11-be0e-8798490c425f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:36.108160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426704b1-1c0f-4d11-be0e-8798490c425f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:36.108438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426704b1-1c0f-4d11-be0e-8798490c425f None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:36.109676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426704b1-1c0f-4d11-be0e-8798490c425f None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:36.111204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426704b1-1c0f-4d11-be0e-8798490c425f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:36.113004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 97/199] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:35 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:36.120021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:36.155577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:36.233259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5652cb-4287-416d-a6c0-dc8b5a88a396 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:36.262795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd4994-b8d0-44a0-a526-2e7e51e5e958 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:36.263080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd4994-b8d0-44a0-a526-2e7e51e5e958 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:36.263359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd4994-b8d0-44a0-a526-2e7e51e5e958 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:36.264896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 98/200] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:26:36.269573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:36.270222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:36.270507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:36.276430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:36.276430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] RBAC: Adding query filter params (role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=54c963c9cd174621ad9e153ca2946c62) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:36.276430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:36.276692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:36.276692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:36.276868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:36.276957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:36.279777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 103/201] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:36 2022] GET /identity/v3/role_assignments?user.id=54c963c9cd174621ad9e153ca2946c62&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:36.301988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:36.303224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:36.421434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 99/202] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:36.851553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:36.950930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:36.961686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090126d5-b0b5-44f3-bd1f-44ba9d0455d3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:36.997646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:36.998282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:36.998558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:36.998823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:37.009643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:37.014802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:37.016180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:37.017070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:37.021841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 104/203] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:37.027505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:37.151739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6ba39-7724-4100-8acc-274f4af4eac9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:37.165224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:37.186325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:37.186941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:37.187215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:37.187483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:37.188196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:37.188503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:37.189910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:37.203013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 100/204] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:37.209007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:37.316594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc21216-7025-4683-b1e1-7d914786f6a7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:37.355940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:37.356630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:37.357333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:37.357333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] PATH_INFO: `/v3/users/7f8856894b404eaba9327c3d486ad0f9` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:37.358332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8856894b404eaba9327c3d486ad0f9)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:37.360079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:37.361824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 105/205] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:37 2022] GET /identity/v3/users/7f8856894b404eaba9327c3d486ad0f9 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:37.367323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:37.499461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc49709-a1b5-44ec-931d-b41d190f2827 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:37.534045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:37.534648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:37.534923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:37.535191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:37.578039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:37.602692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:37.604189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:37.706840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 101/206] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:26:37.712156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:37.752628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:37.854256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49986fb1-0d60-4333-9aa5-3e0824e8b594 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:37.889080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd382baa-bd79-47b7-a604-5f1e62d056fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:37.889725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382baa-bd79-47b7-a604-5f1e62d056fa None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:37.890000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382baa-bd79-47b7-a604-5f1e62d056fa None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:37.890271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382baa-bd79-47b7-a604-5f1e62d056fa None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:37.892196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd382baa-bd79-47b7-a604-5f1e62d056fa None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:37.893709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd382baa-bd79-47b7-a604-5f1e62d056fa None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:37.895706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 106/207] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:37 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:37.901548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:37.994344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:38.020268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a9ab-ebfa-4eb7-8e21-209b0ea84d5f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:38.060341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:38.060983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:38.061226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:38.061547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:38.062271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] RBAC: Adding query filter params (role.id=6c4dc8189192426f8d9b637b0f123425, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e, user.id=7f8856894b404eaba9327c3d486ad0f9) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:38.062624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:38.064663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183ed87-5440-4052-91da-c002205a0222 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:38.075069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 102/208] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:37 2022] GET /identity/v3/role_assignments?user.id=7f8856894b404eaba9327c3d486ad0f9&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=6c4dc8189192426f8d9b637b0f123425 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:38.440256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cd3f-efc4-4ffb-93c1-c8b9e7e6cd59 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:38.440776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cd3f-efc4-4ffb-93c1-c8b9e7e6cd59 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:38.440776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cd3f-efc4-4ffb-93c1-c8b9e7e6cd59 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:38.442189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 107/209] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:38.448214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ecd2f-60f8-42e6-b871-8574c2a13e7f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:38.448514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ecd2f-60f8-42e6-b871-8574c2a13e7f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:38.448770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ecd2f-60f8-42e6-b871-8574c2a13e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:38.470328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ab6ecd2f-60f8-42e6-b871-8574c2a13e7f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:38.471530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ab6ecd2f-60f8-42e6-b871-8574c2a13e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:38.548265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 103/210] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:38.561911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9510-a707-4e4e-9382-47e30f1e0aad None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:38.562197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9510-a707-4e4e-9382-47e30f1e0aad None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:38.562453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9510-a707-4e4e-9382-47e30f1e0aad None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:38.564386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 108/211] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:38.569974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:38.570259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:38.570531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:38.588999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:38.590190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:38.655818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:38.664266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 104/212] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:38.848049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f9510-a707-4e4e-9382-47e30f1e0aad None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:38.930468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:38.969494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f9510-a707-4e4e-9382-47e30f1e0aad None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:38.979023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.012980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4ceee-3619-44f9-b825-bde0649525fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.014030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4ceee-3619-44f9-b825-bde0649525fe None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.014316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4ceee-3619-44f9-b825-bde0649525fe None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.014591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4ceee-3619-44f9-b825-bde0649525fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.028768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4ceee-3619-44f9-b825-bde0649525fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-ccb4ceee-3619-44f9-b825-bde0649525fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:39.035484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:39.036658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:39.037449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 109/213] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:39.038483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902ea0-1375-4e61-8983-33a5f0a069ea None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.038576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902ea0-1375-4e61-8983-33a5f0a069ea None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.038840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902ea0-1375-4e61-8983-33a5f0a069ea None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.040226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 110/214] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:39.040809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfef530-02f7-4308-b394-f4917764edd1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.043265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902ea0-1375-4e61-8983-33a5f0a069ea None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.078180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e0278-ebeb-4b7d-b746-a663637ea33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.079443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e0278-ebeb-4b7d-b746-a663637ea33c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.079717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e0278-ebeb-4b7d-b746-a663637ea33c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.079984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e0278-ebeb-4b7d-b746-a663637ea33c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.090248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e0278-ebeb-4b7d-b746-a663637ea33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-034e0278-ebeb-4b7d-b746-a663637ea33c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:39.095119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:39.096214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:39.097016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 105/215] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:26:39.099841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844089ad-307b-4fe6-8b0b-129503324132 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.100068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844089ad-307b-4fe6-8b0b-129503324132 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.100388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844089ad-307b-4fe6-8b0b-129503324132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.119413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-844089ad-307b-4fe6-8b0b-129503324132 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:39.120650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-844089ad-307b-4fe6-8b0b-129503324132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:39.145843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902ea0-1375-4e61-8983-33a5f0a069ea None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.173245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.178704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.181281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.181281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.181281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.181281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:39.181281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.182192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.186234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 106/216] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:39.195692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52bd1a-d573-42aa-8226-4af5092949c6 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.195692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52bd1a-d573-42aa-8226-4af5092949c6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.195692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52bd1a-d573-42aa-8226-4af5092949c6 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.195692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 107/217] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:39.195692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52bd1a-d573-42aa-8226-4af5092949c6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.196025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 111/218] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:39.196761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.207621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.302236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a153b4-6ee2-45fb-9f58-fa2463115996 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.302549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52bd1a-d573-42aa-8226-4af5092949c6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.336596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.336770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.337226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.337327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.337500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.337605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.337766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] PATH_INFO: `/v3/users/ac4bb978d94e4e72bee40bc247a92609` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.337869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.338465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:39.338737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.340107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c1ef5-cf52-4cbf-80fe-5b70686d8021 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.349438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 108/219] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:39.350502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.350731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.351005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.352541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bb978d94e4e72bee40bc247a92609)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.353982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.360456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 112/220] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/users/ac4bb978d94e4e72bee40bc247a92609 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:39.360456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.373005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:39.374198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:39.420930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.428119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.442433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 109/221] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:39.443082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.472974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747dfb8-f275-41b1-936b-75d76c739043 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.510374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.511001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.511280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.511550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] PATH_INFO: `/v3/users/54c963c9cd174621ad9e153ca2946c62` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.512701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] RBAC: Authorizing `identity:get_user(user_id=54c963c9cd174621ad9e153ca2946c62)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.514151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.516676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 113/222] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/users/54c963c9cd174621ad9e153ca2946c62 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:39.521084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.548997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138535-4620-4025-9b6d-777d56e42cf8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.586436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.587367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.587709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.588053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.628210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d73c-4c35-4512-9468-fa97a4a7fee8 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.637830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.658496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.659980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.663553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.667726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.667726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.667726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.667726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.667726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.672932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 114/223] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:39.672932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.676571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.762925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 110/224] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:39.762925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.780690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f66a8-ecb8-434a-ac84-2980a44135ec None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.814599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.815227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.815505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.815776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.816921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.818377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.820157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 115/225] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:39.820849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:39.846866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.865933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9fac-e4a0-4144-a1b6-3a19d2f46423 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.891499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:39.900342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.900960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.901235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.903386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.903386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:39.903572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:39.921178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab56f41-0550-4361-a780-9ef231f664a8 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:39.955490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:39.956110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:39.956417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:39.956682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:39.966710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:39.972484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:39.973453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:39.974255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 116/226] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:39.974924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.008471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 111/227] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:39 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:40.009291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.092123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41b6eb-255e-4850-9bc4-f7dae21141ad None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.123422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:40.124383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcebf77-b13e-4ca3-bd32-aa729522ac64 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.130368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.131124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.131310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.131580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.132816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.140290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:40.140290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 117/228] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:39 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:40.140290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.169954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.170584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.170869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.171150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/54c963c9cd174621ad9e153ca2946c62/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.173688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] RBAC: Authorizing `identity:create_grant(project_id=994dc5a0b7fe4fdebc1ce89413258472, user_id=54c963c9cd174621ad9e153ca2946c62, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.175137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:40.178855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:40.248076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464a79e-d32c-4d98-8ce7-2660121eb999 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.249903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 112/229] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:40 2022] PUT /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/54c963c9cd174621ad9e153ca2946c62/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:40.250928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.312357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.313022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.313835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.313835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.314313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:40.314786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.317144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:40.328337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 118/230] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:40.336591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.396731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25396af-8b65-4d19-bc61-ea99a8f4f4bd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.437049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:40.448832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.449726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.450006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.450283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.451037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=ac4bb978d94e4e72bee40bc247a92609) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:40.451348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.452928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:40.459536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 113/231] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:40 2022] GET /identity/v3/role_assignments?user.id=ac4bb978d94e4e72bee40bc247a92609&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:40.492445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79aa9e-8129-4bca-8ef6-65da75e1198f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.534800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.535840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.536348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.536885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] PATH_INFO: `/v3/users/54c963c9cd174621ad9e153ca2946c62` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.538272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] RBAC: Authorizing `identity:get_user(user_id=54c963c9cd174621ad9e153ca2946c62)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.539937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:40.541771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 119/232] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:40 2022] GET /identity/v3/users/54c963c9cd174621ad9e153ca2946c62 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:40.548405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.625385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:40.685023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065964be-e514-4e1e-bb0b-3ea7da53e858 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.731477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17e9c9-1f30-402a-b809-cf03545fa826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.732510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17e9c9-1f30-402a-b809-cf03545fa826 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.732990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17e9c9-1f30-402a-b809-cf03545fa826 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.733455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17e9c9-1f30-402a-b809-cf03545fa826 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.735361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17e9c9-1f30-402a-b809-cf03545fa826 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.737190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17e9c9-1f30-402a-b809-cf03545fa826 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:40.739343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 114/233] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:40 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:40.745171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:40.862843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15caeee-c301-4493-87e2-0b1c7d821c0f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:40.903084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:40.914680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b97db-5102-48d0-923c-c87073bef276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:40.915348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b97db-5102-48d0-923c-c87073bef276 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:40.915627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b97db-5102-48d0-923c-c87073bef276 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:40.915901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b97db-5102-48d0-923c-c87073bef276 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/54c963c9cd174621ad9e153ca2946c62/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:40.923845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b97db-5102-48d0-923c-c87073bef276 None admin] RBAC: Authorizing `identity:create_grant(project_id=994dc5a0b7fe4fdebc1ce89413258472, user_id=54c963c9cd174621ad9e153ca2946c62, role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:40.923845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b97db-5102-48d0-923c-c87073bef276 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:41.063352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 120/234] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:40 2022] PUT /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/54c963c9cd174621ad9e153ca2946c62/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:42.722899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf8b1b-f557-4158-829a-bb5e0ea43403 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:42.723437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf8b1b-f557-4158-829a-bb5e0ea43403 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:42.723437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf8b1b-f557-4158-829a-bb5e0ea43403 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:42.725321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 115/235] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:42.726819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5f9a1-a162-4aff-b22e-da11f9a3da52 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:42.727200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5f9a1-a162-4aff-b22e-da11f9a3da52 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:42.727368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5f9a1-a162-4aff-b22e-da11f9a3da52 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:42.728869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 121/236] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:42.731574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd843de-9ae0-4ef5-9b56-2a9db67fdd6c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:42.731953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd843de-9ae0-4ef5-9b56-2a9db67fdd6c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:42.732353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd843de-9ae0-4ef5-9b56-2a9db67fdd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:42.736518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c365e57-ee22-45c5-b2e6-600753a57916 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:42.740191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c365e57-ee22-45c5-b2e6-600753a57916 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:42.740560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c365e57-ee22-45c5-b2e6-600753a57916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:42.766995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bdd843de-9ae0-4ef5-9b56-2a9db67fdd6c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:42.768253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bdd843de-9ae0-4ef5-9b56-2a9db67fdd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:42.781396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2c365e57-ee22-45c5-b2e6-600753a57916 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:42.782724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2c365e57-ee22-45c5-b2e6-600753a57916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:42.905553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 116/237] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:42.923457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 122/238] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:43.104268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100f731-2845-4a79-a4ed-3dd1ffc5d5d0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.104620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100f731-2845-4a79-a4ed-3dd1ffc5d5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.104826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100f731-2845-4a79-a4ed-3dd1ffc5d5d0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.106447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 117/239] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:43.111394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.111686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.111955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.137523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:43.138947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:43.220662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100f731-2845-4a79-a4ed-3dd1ffc5d5d0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:43.233628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 123/240] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:43.294043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:43.318493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:43.359451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:43.470860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49623be-cadc-4397-95c8-009ce1d0923b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:43.477358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100f731-2845-4a79-a4ed-3dd1ffc5d5d0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:43.508878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaf9b7-af48-4755-8339-09c55b7f1ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:43.509521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaf9b7-af48-4755-8339-09c55b7f1ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.509804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaf9b7-af48-4755-8339-09c55b7f1ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.510120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaf9b7-af48-4755-8339-09c55b7f1ee1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.511389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaf9b7-af48-4755-8339-09c55b7f1ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:43.513014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaf9b7-af48-4755-8339-09c55b7f1ee1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:43.514705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 124/241] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:43.516867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d11ca-ee33-465c-b61d-16c50a974cf2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.517145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d11ca-ee33-465c-b61d-16c50a974cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.517415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d11ca-ee33-465c-b61d-16c50a974cf2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.518660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 125/242] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:43.521788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327801a-6ae4-4176-90c8-23f8d94566cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:43.522693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327801a-6ae4-4176-90c8-23f8d94566cf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.523166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327801a-6ae4-4176-90c8-23f8d94566cf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.523474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d11ca-ee33-465c-b61d-16c50a974cf2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:43.523835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327801a-6ae4-4176-90c8-23f8d94566cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.539313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327801a-6ae4-4176-90c8-23f8d94566cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e327801a-6ae4-4176-90c8-23f8d94566cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:43.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:43.545358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:43.546327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 118/243] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:43.548040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.548040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.548240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.565195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:43.566384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:43.652259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 119/244] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:43.652259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:43.659059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d11ca-ee33-465c-b61d-16c50a974cf2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:43.679454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:43.705228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:43.705932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.706520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.706881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.708050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:43.710028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:43.711493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:43.769051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c8421-5693-4557-8f5d-11b61755b677 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:43.800699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 126/245] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:43 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:43.802351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:43.818120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:43.819152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.819152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.819450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.845499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:43.851849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:43.852783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:43.853597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 120/246] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:43.860342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:43.942446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ebf2d-d0e6-4b9a-9dbd-902364844afb None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:43.988569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:43.989249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:43.989534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:43.989806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:43.990497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:43.990830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:43.992287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06d20a-452d-44ad-a625-ce21150a661e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.001787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 127/247] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:44.003093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c49e-2726-4753-ba4f-3c5f42599c86 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.003379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c49e-2726-4753-ba4f-3c5f42599c86 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.003652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c49e-2726-4753-ba4f-3c5f42599c86 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.005141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 128/248] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:44.005654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c49e-2726-4753-ba4f-3c5f42599c86 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.013259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc1077-079f-4036-85f6-9aebde25669b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.026098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.046974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.056074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.056756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.057097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.057371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.058036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:44.059324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:44.061024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.070968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 121/249] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:44.070968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.121040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c49e-2726-4753-ba4f-3c5f42599c86 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.168931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84b82f-67c8-4dfe-a54d-e3bfe17e3ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.169588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b82f-67c8-4dfe-a54d-e3bfe17e3ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.169860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b82f-67c8-4dfe-a54d-e3bfe17e3ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.170125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b82f-67c8-4dfe-a54d-e3bfe17e3ed7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.183055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84b82f-67c8-4dfe-a54d-e3bfe17e3ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-7f84b82f-67c8-4dfe-a54d-e3bfe17e3ed7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:44.189752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:44.190657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:44.191434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:44.195851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 129/250] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:44.197062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b27e61-1cef-4b6b-b33e-77a04f4da49e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.197186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.197433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.197709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.218841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:44.220559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:44.243191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.243838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.244116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.244411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] PATH_INFO: `/v3/users/b65fbecabb7f4b1bbe588b9b823192c3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.261062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=b65fbecabb7f4b1bbe588b9b823192c3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:44.264533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.264533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 122/251] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/users/b65fbecabb7f4b1bbe588b9b823192c3 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:44.265152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.296599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.312865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 130/252] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:44.316179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.353246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.456173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4674222-beb8-4d6b-b10c-c801e8d7b8a6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.491311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78532a11-859c-4f98-892d-1504d3d801aa None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.522951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.523829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.524119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.524631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] PATH_INFO: `/v3/users/54c963c9cd174621ad9e153ca2946c62` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.527036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] RBAC: Authorizing `identity:get_user(user_id=54c963c9cd174621ad9e153ca2946c62)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:44.528531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.531988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 123/253] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/users/54c963c9cd174621ad9e153ca2946c62 => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:44.534730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.549759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.550369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.550657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.550920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.551703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:44.552010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:44.553485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.569831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 131/254] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:44.570460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.704372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.710320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef37f0-901a-421d-982d-3d54aa9f7660 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.712863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.729229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bfc4a-f71a-4887-8e36-5d84cc59056b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:44.746077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.746699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.746974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.747247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.770759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:44.771380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:44.771658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:44.771937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:44.819031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:44.820538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.851131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:44.852753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:44.918032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.925544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 124/255] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:44.926564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:44.952326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:44.974350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 132/256] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:44.975237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.038794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cfa3-78fa-4a88-a57e-96a9f05adca6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.073183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.073819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.074093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.074363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] PATH_INFO: `/v3/users/ac4bb978d94e4e72bee40bc247a92609` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.075483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bb978d94e4e72bee40bc247a92609)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.077027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.080579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 125/257] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/users/ac4bb978d94e4e72bee40bc247a92609 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.080579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.088754 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74583-f9dd-4e03-b6b7-f3ced2bf11f7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.126745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.127475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.127756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.128026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.139148 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:45.143299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:45.145926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:45.146993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 133/258] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:45.148033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.182287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd33172-d8b5-4aa6-8e5e-8ca335afe8ec None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.204085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:45.218099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.218923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.219731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.220130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.221299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.222829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.224661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 126/259] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.225452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.267692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf2757-85eb-47df-9f79-5694f1538233 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.306489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.307493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.307914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.308349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.309619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.311492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.313473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 134/260] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.314527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.330525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fa37b-e863-4249-9350-3b39ba80f4b7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.362090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:45.367343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.368122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.375489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.375489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.375489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.375489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.375489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 127/261] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.375489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.429387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4648fbb-6df5-47bb-8448-acac5c8e438e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.445572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:45.468308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.469236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.469682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.470097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.470896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:45.471442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.473286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.478201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7b90-3c66-41ee-bdab-03bcb433ae6a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.482180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 135/262] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.486945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.513353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.514116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.514492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.514906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.516590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] RBAC: Adding query filter params (role.id=6c4dc8189192426f8d9b637b0f123425, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=b65fbecabb7f4b1bbe588b9b823192c3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:45.517002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.518594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.524601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 128/263] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/role_assignments?user.id=b65fbecabb7f4b1bbe588b9b823192c3&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=6c4dc8189192426f8d9b637b0f123425 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.525466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.595025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:45.609095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf86099-ab49-40e1-8759-a54f588e83e1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.650948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.652122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.652735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.653203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.654154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=54c963c9cd174621ad9e153ca2946c62) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:45.654665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.657158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.664361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 136/264] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/role_assignments?user.id=54c963c9cd174621ad9e153ca2946c62&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.665596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.669316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b65e69-d9a1-4a83-88cb-c1e86c9c468b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.707912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.709147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.709147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.709355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/ac4bb978d94e4e72bee40bc247a92609/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.711362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=994dc5a0b7fe4fdebc1ce89413258472, user_id=ac4bb978d94e4e72bee40bc247a92609, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.713005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.718407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:45.789000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f02b5-f1b6-4dd2-b830-aaf9e26af829 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:45.813602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 129/265] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:45 2022] PUT /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/ac4bb978d94e4e72bee40bc247a92609/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:45.832899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:45.834038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:45.834593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:45.835111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] PATH_INFO: `/v3/users/54c963c9cd174621ad9e153ca2946c62` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:45.836799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] RBAC: Authorizing `identity:get_user(user_id=54c963c9cd174621ad9e153ca2946c62)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:45.838538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:45.840862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 137/266] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/users/54c963c9cd174621ad9e153ca2946c62 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:45.847449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:45.983280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8f1e-0adc-4e6f-978b-d287e22a59d5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:46.012396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:46.024664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:46.031578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.031578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.031578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.107396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:46.109815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:46.227893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 130/267] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:46.234156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:46.386741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d5c2d-cedf-4c4a-af4a-7cd7c38e1b19 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:46.400927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:46.440255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e452a10-1be2-447a-83d1-0f38fc9ae642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:46.440896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e452a10-1be2-447a-83d1-0f38fc9ae642 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.441213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e452a10-1be2-447a-83d1-0f38fc9ae642 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.441504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e452a10-1be2-447a-83d1-0f38fc9ae642 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.448919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e452a10-1be2-447a-83d1-0f38fc9ae642 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:46.450419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e452a10-1be2-447a-83d1-0f38fc9ae642 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:46.452119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 138/268] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:46 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:46.458339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:46.475734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aec38-f307-493e-b6bc-263375e02575 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.476031 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aec38-f307-493e-b6bc-263375e02575 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.476321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aec38-f307-493e-b6bc-263375e02575 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.479258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 139/269] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:46.485876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2be48b-e53d-4599-bfca-1fd68febad26 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.486158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2be48b-e53d-4599-bfca-1fd68febad26 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.486417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2be48b-e53d-4599-bfca-1fd68febad26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.516313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:46.527607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5b2be48b-e53d-4599-bfca-1fd68febad26 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:46.528859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5b2be48b-e53d-4599-bfca-1fd68febad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:46.613124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9825a4-1c80-4ce3-8590-dec6fb648a06 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:46.648087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:46.648756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.649037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.649405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.650127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] RBAC: Adding query filter params (role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=54c963c9cd174621ad9e153ca2946c62) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:46.650434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:46.651879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:46.653870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 140/270] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:46.655115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbb0c3-6c0a-46f6-990d-4543981100b1 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.655493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbb0c3-6c0a-46f6-990d-4543981100b1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.655765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbb0c3-6c0a-46f6-990d-4543981100b1 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.661623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 141/271] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:46.663217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:46.663495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:46.663760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:46.663946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 131/272] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:46 2022] GET /identity/v3/role_assignments?user.id=54c963c9cd174621ad9e153ca2946c62&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:46.683512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:46.684823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:46.757660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 142/273] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:47.046405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:47.057069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:47.102711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:47.164573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:47.174741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f746-5324-4032-9f03-bc84636a25b7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:47.186195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc4c02-b261-4d8b-9072-f5d607aba81c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:47.216864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:47.217508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:47.217832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:47.218042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:47.229866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:47.231798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:47.232539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:47.232908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:47.233247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:47.234983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:47.235511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:47.236389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:47.237160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:47.237745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:47.237819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 132/274] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:47.241604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 143/275] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:47.244012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:47.244124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:47.347863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b8461-9f70-46a5-b6d5-1fe4e37c9b1c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:47.350592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:47.388339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:47.388947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:47.389269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:47.389545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:47.390202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:47.390523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:47.391294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f404394-507f-4b9b-8ad8-764f1c557c33 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:47.392171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:47.405659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 144/276] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:47.406712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:47.447040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aa499-9e85-4251-b38d-59362709c51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:47.448001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa499-9e85-4251-b38d-59362709c51b None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:47.448530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa499-9e85-4251-b38d-59362709c51b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:47.448998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa499-9e85-4251-b38d-59362709c51b None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:47.449846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aa499-9e85-4251-b38d-59362709c51b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:47.452035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aa499-9e85-4251-b38d-59362709c51b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:47.516413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b09ed5-4c4f-4b3c-a488-80e215276bdb None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:47.551756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:47.552415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:47.552692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:47.552959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] PATH_INFO: `/v3/users/1c2c1b8bbf284fb296eb69c232ce9c55` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:47.567380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] RBAC: Authorizing `identity:get_user(user_id=1c2c1b8bbf284fb296eb69c232ce9c55)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:47.568872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:47.575128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 145/277] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:47 2022] GET /identity/v3/users/1c2c1b8bbf284fb296eb69c232ce9c55 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:47.576278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:47.583554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 133/278] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:26:47 2022] POST /identity/v3/users => generated 293 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:47.615268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:47.703790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd24c7-b206-49ca-84de-17d1cfc6286d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:47.744178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:47.745154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:47.745659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:47.746120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:47.747194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, user.id=1c2c1b8bbf284fb296eb69c232ce9c55, scope.system=all) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:47.747703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:47.749573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f6edd-cb10-47fa-9df8-10479f9691bf None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:47.755847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 146/279] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Sep 7 14:26:47 2022] GET /identity/v3/role_assignments?user.id=1c2c1b8bbf284fb296eb69c232ce9c55&scope.system=all&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:48.055672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207a72d-2d4d-4e10-8ded-7d159c06a31c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:48.056638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207a72d-2d4d-4e10-8ded-7d159c06a31c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:48.057112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207a72d-2d4d-4e10-8ded-7d159c06a31c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:48.058683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 134/280] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:48.064184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eadf6-c79d-472b-8fbf-da795303e1fd None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:48.064706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eadf6-c79d-472b-8fbf-da795303e1fd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:48.065175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eadf6-c79d-472b-8fbf-da795303e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:48.084694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e82eadf6-c79d-472b-8fbf-da795303e1fd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:48.086291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e82eadf6-c79d-472b-8fbf-da795303e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:48.160922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 147/281] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:48.484217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207a72d-2d4d-4e10-8ded-7d159c06a31c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:48.488878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:48.588985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646219a-b9d8-4e84-81cd-29b92186367f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:48.589512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646219a-b9d8-4e84-81cd-29b92186367f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:48.589512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646219a-b9d8-4e84-81cd-29b92186367f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:48.591104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 148/282] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:48.597258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:48.597696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:48.597800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:48.617269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:48.618464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:48.618660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207a72d-2d4d-4e10-8ded-7d159c06a31c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:48.636513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:48.679828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:48.680497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:48.680772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:48.681041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:48.701755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:48.707813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:48.708740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:48.709536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 135/283] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:48.716453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:48.751743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 149/284] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:48.814115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:48.869350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788ed7c-9380-4a1f-82c2-fc25b84d4a48 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:48.916932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:48.917911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:48.918392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:48.918890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:48.919756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:48.920355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:48.921970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:48.935117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 136/285] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:48.941175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.094249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1149b90-e5d4-41a9-9a9a-b400760bcdf5 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:49.118632 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.139143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:49.140081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:49.140846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:49.141322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] PATH_INFO: `/v3/users/b65fbecabb7f4b1bbe588b9b823192c3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:49.148883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] RBAC: Authorizing `identity:get_user(user_id=b65fbecabb7f4b1bbe588b9b823192c3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:49.150580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:49.152435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 150/286] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:48 2022] GET /identity/v3/users/b65fbecabb7f4b1bbe588b9b823192c3 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:49.158672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.187982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:49.242411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359af66-5c75-4510-98be-7d22847aafd7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:49.272135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:49.287686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:49.288367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:49.288645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:49.288916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:49.316360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:49.321833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:49.322719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:49.323503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 137/287] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:26:49.330614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.362252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e2445-144c-4340-b76d-1f50cabac11f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:49.430132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:49.430770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:49.431047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:49.431324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:49.432604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:49.434049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:49.435694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 151/288] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:49 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:49.442002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.527854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:49.530193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d988b92-c540-4729-83d6-cda9958d67fc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:49.544556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bb9ae-e177-4ba0-af54-b82b9658bf69 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:49.579417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:49.580080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:49.580391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:49.580665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/b65fbecabb7f4b1bbe588b9b823192c3/roles/6c4dc8189192426f8d9b637b0f123425` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:49.582593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] RBAC: Authorizing `identity:create_grant(project_id=994dc5a0b7fe4fdebc1ce89413258472, user_id=b65fbecabb7f4b1bbe588b9b823192c3, role_id=6c4dc8189192426f8d9b637b0f123425)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:49.584316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:49.603203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:49.603885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:49.604183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:49.604634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:49.605312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:49.605616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:49.607001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:49.615850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 138/289] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:49.625108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.663372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 152/290] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:26:49 2022] PUT /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/users/b65fbecabb7f4b1bbe588b9b823192c3/roles/6c4dc8189192426f8d9b637b0f123425 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:26:49.699775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:49.773509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c398-9e64-470e-89eb-76ea864022e1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:49.827713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:49.828940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:49.829222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:49.829487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] PATH_INFO: `/v3/users/ac4bb978d94e4e72bee40bc247a92609` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:49.830637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bb978d94e4e72bee40bc247a92609)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:49.832098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:49.834038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 139/291] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:49 2022] GET /identity/v3/users/ac4bb978d94e4e72bee40bc247a92609 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:49.839656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:49.961579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217295c-deaf-4f8f-a576-71e67253a13e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:50.009636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e0da6-be7f-43db-ad0b-17ea9df755c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:50.010249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e0da6-be7f-43db-ad0b-17ea9df755c0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.010519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e0da6-be7f-43db-ad0b-17ea9df755c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.010786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e0da6-be7f-43db-ad0b-17ea9df755c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.029432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:50.105635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e0da6-be7f-43db-ad0b-17ea9df755c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:50.107159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e0da6-be7f-43db-ad0b-17ea9df755c0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:50.248521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 153/292] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:50.252076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:50.354646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3caa93-413d-4eda-b0d4-3799b2274b11 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.355304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3caa93-413d-4eda-b0d4-3799b2274b11 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.355781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3caa93-413d-4eda-b0d4-3799b2274b11 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.357656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 154/293] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:50.363320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb6dcd-bf1d-4f1c-b72f-997f77db13d6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.363839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb6dcd-bf1d-4f1c-b72f-997f77db13d6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.364380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb6dcd-bf1d-4f1c-b72f-997f77db13d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.370342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023834ab-5515-4d26-b429-d4b547a80e4c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:50.384263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-56fb6dcd-bf1d-4f1c-b72f-997f77db13d6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:50.387885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-56fb6dcd-bf1d-4f1c-b72f-997f77db13d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:50.411103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:50.412049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.412365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.412639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.414114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:50.415987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:50.418423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 140/294] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:50 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:50.420473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:50.429170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:50.445748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:50.502427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 155/295] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:50.567109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5d8cf-4544-41c7-9aa7-916273759f54 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.567385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5d8cf-4544-41c7-9aa7-916273759f54 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.567657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5d8cf-4544-41c7-9aa7-916273759f54 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.569614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c50b8-5225-473a-b9dc-8259bfea7b03 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:50.570436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 156/296] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:50.574337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebda7d-d3a2-4845-a69f-3a6d2c957257 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.574613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebda7d-d3a2-4845-a69f-3a6d2c957257 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.574882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebda7d-d3a2-4845-a69f-3a6d2c957257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.595829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cbebda7d-d3a2-4845-a69f-3a6d2c957257 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:50.597323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cbebda7d-d3a2-4845-a69f-3a6d2c957257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:50.605484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:50.606356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.606799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.607171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.608117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=ac4bb978d94e4e72bee40bc247a92609) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:50.608546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:50.611453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf7d71-2ff7-4465-936a-aedf6a105d06 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:50.620869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 141/297] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:50 2022] GET /identity/v3/role_assignments?user.id=ac4bb978d94e4e72bee40bc247a92609&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:50.680343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 157/298] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:26:50.685357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f37d43-5f7a-4fca-a791-18979b403c60 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.685640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f37d43-5f7a-4fca-a791-18979b403c60 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.685912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f37d43-5f7a-4fca-a791-18979b403c60 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.687616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 142/299] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:50.702166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8737-d5b6-4158-8968-46358f29815c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:50.702482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8737-d5b6-4158-8968-46358f29815c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:50.702705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8737-d5b6-4158-8968-46358f29815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:50.728726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-257e8737-d5b6-4158-8968-46358f29815c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:50.729980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-257e8737-d5b6-4158-8968-46358f29815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:50.824685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 158/300] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:50.873219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f37d43-5f7a-4fca-a791-18979b403c60 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:50.934400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.001882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f37d43-5f7a-4fca-a791-18979b403c60 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.037025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.037634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.037906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.038171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.051238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:51.056782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:51.057663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:51.058459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 143/301] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:51.065711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8737-d5b6-4158-8968-46358f29815c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.076126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.114082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.185571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8737-d5b6-4158-8968-46358f29815c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.197823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82805785-93f9-4e43-88c6-e3c48c76084a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.206403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.222263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.222870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.223157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.223412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.224074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:51.224403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.225783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.231390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.232019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.232326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.232609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.234981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 159/302] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:51.235568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.247501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:51.253813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:51.255330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 144/303] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:26:51.256668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.349396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.354997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3808-1199-43fe-b497-32161e723be7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.365346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae848d8-3724-4e80-b86f-2c31a9976485 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.390482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.391460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.391737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.392007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.393152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.394623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.396197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 160/304] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:51.396960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.405133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.406554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.407222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.407566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] PATH_INFO: `/v3/users/1c2c1b8bbf284fb296eb69c232ce9c55` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.408726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] RBAC: Authorizing `identity:get_user(user_id=1c2c1b8bbf284fb296eb69c232ce9c55)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.410455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.416463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 145/305] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/users/1c2c1b8bbf284fb296eb69c232ce9c55 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:51.416463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.452135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.507025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3ec4e-400c-47e7-8cc8-5873fc2eb9fa None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.521977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a129-6e36-4a6c-81ad-30139a7e3710 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.542574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.543179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.543455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.543722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.544420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:51.544717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.546377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.559248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 161/306] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:51.559248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.561227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.561846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.562125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.562950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] PATH_INFO: `/v3/groups` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.563575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.565027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.586729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.633578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 146/307] 10.222.0.57 () {60 vars in 1134 bytes} [Wed Sep 7 14:26:51 2022] POST /identity/v3/groups => generated 229 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:26:51.634678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.689011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6446-25f3-4ea1-9d3e-c557bd2cf09a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.742111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d079f-a828-4deb-adab-98933d554332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.743792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d079f-a828-4deb-adab-98933d554332 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.744577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d079f-a828-4deb-adab-98933d554332 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.745086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d079f-a828-4deb-adab-98933d554332 None admin] PATH_INFO: `/v3/system/users/1c2c1b8bbf284fb296eb69c232ce9c55/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.747849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d079f-a828-4deb-adab-98933d554332 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c2c1b8bbf284fb296eb69c232ce9c55, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.750241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:51.750718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d079f-a828-4deb-adab-98933d554332 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.756491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d32e-1cf1-40f0-abb9-94ee0b1a6e0e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.796157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.797419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.797695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.798229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] PATH_INFO: `/v3/users/38171fe8d0534c2d94cb60eb55923a2e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.813363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] RBAC: Authorizing `identity:get_user(user_id=38171fe8d0534c2d94cb60eb55923a2e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.814916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.816802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 147/308] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/users/38171fe8d0534c2d94cb60eb55923a2e => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:51.822327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:51.823252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 162/309] 10.222.0.57 () {58 vars in 1334 bytes} [Wed Sep 7 14:26:51 2022] PUT /identity/v3/system/users/1c2c1b8bbf284fb296eb69c232ce9c55/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:26:51.943906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154d89-4000-4b75-b73a-15f5b9efded1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:51.982464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:51.983085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:51.983360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:51.983679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:51.985022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] RBAC: Adding query filter params (role.id=6c4dc8189192426f8d9b637b0f123425, user.id=38171fe8d0534c2d94cb60eb55923a2e, scope.system=all) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:51.985333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:51.986889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af84a2c-03c8-4741-bb7e-68aaa6c91ced None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:51.993029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 148/310] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Sep 7 14:26:51 2022] GET /identity/v3/role_assignments?user.id=38171fe8d0534c2d94cb60eb55923a2e&scope.system=all&role.id=6c4dc8189192426f8d9b637b0f123425 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:52.327126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bcab-4069-4c64-81cf-65f04833388f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:52.327512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bcab-4069-4c64-81cf-65f04833388f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:52.327693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bcab-4069-4c64-81cf-65f04833388f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:52.329516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 163/311] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:52.334833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:52.335295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:52.335371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:52.354474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:52.355662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:52.370254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:52.431308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 149/312] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:52.695336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bcab-4069-4c64-81cf-65f04833388f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:52.793958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:52.804564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bcab-4069-4c64-81cf-65f04833388f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:52.840077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:52.840989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:52.841472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:52.841922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:52.852922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:52.858101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:52.858995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:52.859805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:52.861742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 164/313] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:52.868240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:52.974906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e5f1-84f7-45dc-8056-ffc43cf2e315 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:53.013506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:53.014109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:53.014382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:53.014650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:53.015866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:53.016192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:53.017566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:53.030303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 150/314] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:53.033224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:53.133768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d8bb0-d3b0-4f3a-b395-2fa804871d41 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:53.173560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:53.174566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:53.175045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:53.175584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] PATH_INFO: `/v3/users/b65fbecabb7f4b1bbe588b9b823192c3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:53.177010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] RBAC: Authorizing `identity:get_user(user_id=b65fbecabb7f4b1bbe588b9b823192c3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:53.178669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:53.180836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 165/315] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:53 2022] GET /identity/v3/users/b65fbecabb7f4b1bbe588b9b823192c3 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:53.189498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:53.204805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:53.297623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82440d-3006-47de-93c9-05507cbe41dd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:53.336075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:53.336724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:53.336978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:53.337247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:53.406399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:53.407866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:53.425492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:53.506497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 151/316] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:53.512077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:53.532819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:53.615730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfef87-d002-48c7-a9a3-0e8833b10964 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:53.652398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea27f2f-30fc-488e-9c03-64a6f5acf3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:53.653034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27f2f-30fc-488e-9c03-64a6f5acf3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:53.653309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27f2f-30fc-488e-9c03-64a6f5acf3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:53.653578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27f2f-30fc-488e-9c03-64a6f5acf3b7 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:53.654710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea27f2f-30fc-488e-9c03-64a6f5acf3b7 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:53.656827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea27f2f-30fc-488e-9c03-64a6f5acf3b7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:53.658767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 166/317] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:53 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:53.664625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:53.767875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b709c-a195-47c3-ad88-50042cb231eb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:53.778686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:53.802322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:53.804273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:53.804273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:53.804273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:53.804273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] RBAC: Adding query filter params (role.id=6c4dc8189192426f8d9b637b0f123425, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472, user.id=b65fbecabb7f4b1bbe588b9b823192c3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:53.804637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:53.806068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f99fcf-3037-4f79-ab03-f9a750b69281 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:53.813128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 152/318] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:26:53 2022] GET /identity/v3/role_assignments?user.id=b65fbecabb7f4b1bbe588b9b823192c3&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=6c4dc8189192426f8d9b637b0f123425 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:54.016381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3efa30-a1e3-4604-b99c-53183f155611 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.016898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3efa30-a1e3-4604-b99c-53183f155611 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.016898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3efa30-a1e3-4604-b99c-53183f155611 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.018304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 167/319] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:54.024251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c614-25cd-430b-b777-a49742e381cf None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.024557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c614-25cd-430b-b777-a49742e381cf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.024818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c614-25cd-430b-b777-a49742e381cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.045731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3505c614-25cd-430b-b777-a49742e381cf None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:54.047913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3505c614-25cd-430b-b777-a49742e381cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:54.127351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 153/320] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:54.143784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c53d6-383f-4a89-8c06-7f08c9a25cc0 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.144108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c53d6-383f-4a89-8c06-7f08c9a25cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.144374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c53d6-383f-4a89-8c06-7f08c9a25cc0 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.145957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 168/321] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:54.151271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.151560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.151835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.170275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:54.171455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:54.236293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f612b-c7cc-4036-8d78-d40ea265633b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.236581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f612b-c7cc-4036-8d78-d40ea265633b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.236853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f612b-c7cc-4036-8d78-d40ea265633b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.238309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 169/322] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:54.239862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 154/323] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:54.243088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.243368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.243637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.261217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:54.262512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:54.331586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 170/324] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:54.400087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:54.434732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:54.500725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fc51f-434b-4597-8780-f47d934b42c1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:54.508944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:54.527596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:54.538030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:54.538664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.538940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.539210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.540337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:54.541802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:54.543572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 155/325] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:26:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:54.548750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:54.621062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d95ce-469c-4e5a-917c-79c135cca80b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:54.657688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:54.658315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.658595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.658866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.660631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc46b6e-3d29-40a1-9573-882bf621c708 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:54.662140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:54.668792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:54.674329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:54.675364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:54.676172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 171/326] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:54.680940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:54.694716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:54.695325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.695600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.695868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] PATH_INFO: `/v3/groups` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.696522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:54.697941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:54.755649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:54.778163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e8421-b555-45af-b6b8-3c3be23e41e0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:54.802603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 156/327] 10.222.0.57 () {60 vars in 1134 bytes} [Wed Sep 7 14:26:54 2022] POST /identity/v3/groups => generated 226 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:26:54.803471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:54.813517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:54.814182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:54.814462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:54.814720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:54.828169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:54.833322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:54.834494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:54.835292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 172/328] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:54.841045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:54.936602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3b19-71b1-4c7d-8697-ffd233d00165 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:55.006771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f2f8-d6ea-475e-8063-1c5a90efb0ec None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:55.007474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:55.007474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:55.007474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:55.007474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:55.007820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:55.007820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:55.007820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:55.007820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 157/329] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:55.007820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:55.008223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:55.008223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:55.008223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:55.008223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:55.008523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:55.008523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:55.009527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:55.012838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:55.021169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 173/330] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:55.028226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:55.037500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:55.126607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40602-465c-4865-8bd0-dd7065c1d840 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:55.141571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae61dc-a964-4eaf-930f-213b392ad6aa None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:55.172516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:55.173164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:55.173431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:55.173706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] PATH_INFO: `/v3/users/1c2c1b8bbf284fb296eb69c232ce9c55` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:55.174841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] RBAC: Authorizing `identity:get_user(user_id=1c2c1b8bbf284fb296eb69c232ce9c55)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:55.179250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:55.179250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 158/331] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:55 2022] GET /identity/v3/users/1c2c1b8bbf284fb296eb69c232ce9c55 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:55.182112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:55.182750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:55.183023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:55.183292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] PATH_INFO: `/v3/users/38171fe8d0534c2d94cb60eb55923a2e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:55.184671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] RBAC: Authorizing `identity:get_user(user_id=38171fe8d0534c2d94cb60eb55923a2e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:55.185731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:55.185937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:55.188182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 174/332] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:55 2022] GET /identity/v3/users/38171fe8d0534c2d94cb60eb55923a2e => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:26:55.193489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:55.284815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:55.309448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:55.320633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84705917-e9ff-45a0-9aa8-cc90f71c81f6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:55.325202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906f7db-d496-418b-a38a-aecb6937e7bd None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:55.362647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:55.363365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:55.363719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:55.364069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:55.364904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] RBAC: Adding query filter params (role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, user.id=1c2c1b8bbf284fb296eb69c232ce9c55, scope.system=all) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:55.365291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:55.366798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eccd1-9b33-4f6f-9e25-855a3d401dd8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:55.367288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288fd9-a0aa-461d-96e2-755bdf35d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:55.368117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288fd9-a0aa-461d-96e2-755bdf35d2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:55.368766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288fd9-a0aa-461d-96e2-755bdf35d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:55.369195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288fd9-a0aa-461d-96e2-755bdf35d2f4 None admin] PATH_INFO: `/v3/system/users/38171fe8d0534c2d94cb60eb55923a2e/roles/6c4dc8189192426f8d9b637b0f123425` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:55.371523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288fd9-a0aa-461d-96e2-755bdf35d2f4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=38171fe8d0534c2d94cb60eb55923a2e, role_id=6c4dc8189192426f8d9b637b0f123425)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:55.374569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288fd9-a0aa-461d-96e2-755bdf35d2f4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:55.382274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 159/333] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Sep 7 14:26:55 2022] GET /identity/v3/role_assignments?user.id=1c2c1b8bbf284fb296eb69c232ce9c55&scope.system=all&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:55.455105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 175/334] 10.222.0.57 () {58 vars in 1334 bytes} [Wed Sep 7 14:26:55 2022] PUT /identity/v3/system/users/38171fe8d0534c2d94cb60eb55923a2e/roles/6c4dc8189192426f8d9b637b0f123425 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:26:57.481278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54994393-a3d9-40af-91eb-8c5f12b463ea None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:57.481813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54994393-a3d9-40af-91eb-8c5f12b463ea None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:57.481813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54994393-a3d9-40af-91eb-8c5f12b463ea None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:57.483379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 160/335] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:57.488557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d49286-eb84-40c3-977f-feabbc608a54 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:57.488996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d49286-eb84-40c3-977f-feabbc608a54 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:57.489098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d49286-eb84-40c3-977f-feabbc608a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:57.512521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-94d49286-eb84-40c3-977f-feabbc608a54 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:57.513728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-94d49286-eb84-40c3-977f-feabbc608a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:57.601026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 176/336] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:57.630146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954393-47a3-47cf-97db-e478e4133aa6 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:57.630470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954393-47a3-47cf-97db-e478e4133aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:57.630704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954393-47a3-47cf-97db-e478e4133aa6 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:57.632049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 161/337] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:57.637838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:57.638130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:57.638409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:57.665200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:57.666417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:57.733859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:57.751169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 177/338] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:58.028871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2954393-47a3-47cf-97db-e478e4133aa6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:58.094297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:58.201236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2954393-47a3-47cf-97db-e478e4133aa6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:58.296042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95adc6b-4a6f-46d8-b447-426033870ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:58.297483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95adc6b-4a6f-46d8-b447-426033870ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.297763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95adc6b-4a6f-46d8-b447-426033870ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.298038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95adc6b-4a6f-46d8-b447-426033870ea3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.314681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6214fad-90b2-43e3-872c-fc440d46b90e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:58.329298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95adc6b-4a6f-46d8-b447-426033870ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d95adc6b-4a6f-46d8-b447-426033870ea3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:58.334895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:58.335833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:26:58.336713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:58.345300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 162/339] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:26:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:58.346631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebb19e-fa70-4c9e-b843-afdd77e37190 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.346911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebb19e-fa70-4c9e-b843-afdd77e37190 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.347182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebb19e-fa70-4c9e-b843-afdd77e37190 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.348522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 163/340] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:58.355155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.355444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.355720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.389136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:58.390680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:58.399353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:58.403564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:58.403564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.403564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.403564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.428772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:58.434678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:58.435575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 178/341] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:58.436947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:58.508565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:58.526501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 164/342] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:58.527372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:58.622423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4541-f139-49a8-8f27-776030e2f310 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:58.639265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb75af-95fd-4daf-ad5f-bc3f2d646014 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:58.674975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:58.675595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.675868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.676134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.676878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:58.677176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:58.678621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee654d-5b71-4ab6-b07e-ed5c36bac683 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:58.689674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 165/343] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:58.696557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c846aad-235c-4863-918b-e72049df01a0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.696557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c846aad-235c-4863-918b-e72049df01a0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.696557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c846aad-235c-4863-918b-e72049df01a0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.696557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 166/344] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:58.699244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c846aad-235c-4863-918b-e72049df01a0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:58.712107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:58.725389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:58.726515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.727270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.727270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.728622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:58.729128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:58.730797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6dac-e935-4fca-8954-ae849f0e1067 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:58.756625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 179/345] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:26:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:58.761001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.761564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.762023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.796388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:58.797974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:58.819765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c846aad-235c-4863-918b-e72049df01a0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:58.859962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:58.859962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:58.859962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:58.859962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:58.920475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 180/346] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:58.921084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:58.957326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:58.958771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:58.975483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:58.987679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.080489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 167/347] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:26:59.081356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.147546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23b9cf-8128-428f-a148-b577324467f9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.185681 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14953f5a-b4dd-426e-8325-dc1ab0956a4f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.188266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.226523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.227647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.228069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.228801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.238411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.239301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.239784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.240262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.256892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:59.265735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:59.273904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 168/348] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:59.274881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.360688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:59.362669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:59.409458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12830f17-63d8-422f-b6a6-1abdbcf63d3f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.417898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.452931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.453942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.454223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.454496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.456108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:59.458128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:59.461034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 169/349] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:59.461827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.489042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 181/350] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:26:59.489905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.504729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.592799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f980-8ee2-4ccc-b503-4e757cd008cc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.614671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc7b9a-daee-4bcc-aa7a-530552f50e9d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.638745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.639892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.639892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.640345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.641367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:26:59.641773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:59.643165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7541e1e-070c-4ce3-8641-41245693ea1b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:59.652470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 170/351] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:59.654141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03115c06-3ae0-44e1-837b-1a0577868d8e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.654141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03115c06-3ae0-44e1-837b-1a0577868d8e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.654141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03115c06-3ae0-44e1-837b-1a0577868d8e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.655361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 171/352] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:26:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:26:59.657618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03115c06-3ae0-44e1-837b-1a0577868d8e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.664176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.664806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.665080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.665344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.681874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:26:59.687258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:26:59.688223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:26:59.689044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 182/353] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:26:59.689700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.693855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.790376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.800451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03115c06-3ae0-44e1-837b-1a0577868d8e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.809978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc65da-52b6-45ef-bc26-7426e649a200 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:26:59.842087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.842730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.843060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.843368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.853640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:59.855474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:59.855744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d03c15-ac11-4100-ae56-b29c7b22a2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:26:59.856419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d03c15-ac11-4100-ae56-b29c7b22a2ee None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.856700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d03c15-ac11-4100-ae56-b29c7b22a2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.856972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d03c15-ac11-4100-ae56-b29c7b22a2ee None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.858159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d03c15-ac11-4100-ae56-b29c7b22a2ee None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:26:59.859039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 172/354] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:59.860028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.862865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d03c15-ac11-4100-ae56-b29c7b22a2ee None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:26:59.865108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 183/355] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:26:59.866916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:26:59.867299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:26:59.867672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:26:59.891630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:26:59.893146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:26:59.976338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:26:59.981723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 184/356] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:26:59.984509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:26:59.991652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48698-7194-4419-92cc-ff4a5e17b0ad None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.043192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.044857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.044857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.045066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] PATH_INFO: `/v3/users/38171fe8d0534c2d94cb60eb55923a2e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.047083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] RBAC: Authorizing `identity:get_user(user_id=38171fe8d0534c2d94cb60eb55923a2e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:00.048669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:00.051179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 173/357] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/users/38171fe8d0534c2d94cb60eb55923a2e => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:00.051772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.089454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:00.119546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad90745-5794-493e-8132-65b8d2479931 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.162005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.162625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.162897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.163184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.163842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:00.164171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:00.165690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:00.180980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 185/358] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:26:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:00.180980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.191698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faceec55-94a1-4989-b758-a6513cbe5e3d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.242889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.243765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.244671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.245111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.246030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:00.246333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:00.247831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:00.254407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 174/359] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:00.255193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:00.255304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.307346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4794b7-cd4d-4ab8-a60b-0d588ea3d08b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.357315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1acf3-986d-4e26-b197-b5de9c8b6356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.358285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1acf3-986d-4e26-b197-b5de9c8b6356 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.359120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1acf3-986d-4e26-b197-b5de9c8b6356 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.359364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1acf3-986d-4e26-b197-b5de9c8b6356 None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.360878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1acf3-986d-4e26-b197-b5de9c8b6356 None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:00.362226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1acf3-986d-4e26-b197-b5de9c8b6356 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:00.364192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 186/360] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:00.366562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc69f59-1c69-46c0-9640-7d40c48742fa None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.366845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc69f59-1c69-46c0-9640-7d40c48742fa None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.367118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc69f59-1c69-46c0-9640-7d40c48742fa None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.368413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 187/361] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:00.369208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc69f59-1c69-46c0-9640-7d40c48742fa None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.391796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:00.411063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840fee4-d3a4-4c75-b042-201085547619 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.463732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.464753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.465240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.465727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.466460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] RBAC: Adding query filter params (role.id=6c4dc8189192426f8d9b637b0f123425, user.id=38171fe8d0534c2d94cb60eb55923a2e, scope.system=all) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:00.466764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:00.469718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:00.488826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 175/362] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/role_assignments?user.id=38171fe8d0534c2d94cb60eb55923a2e&scope.system=all&role.id=6c4dc8189192426f8d9b637b0f123425 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:00.489032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.531184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc69f59-1c69-46c0-9640-7d40c48742fa None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.572543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:00.583951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.584636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.590334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.590334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.629882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008cab1-3b2c-499c-bdc2-51322328c905 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.675588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd5c04-7366-466e-a19c-d239a39cc5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:00.676284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5c04-7366-466e-a19c-d239a39cc5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.676562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5c04-7366-466e-a19c-d239a39cc5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.676870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5c04-7366-466e-a19c-d239a39cc5c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.690109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd5c04-7366-466e-a19c-d239a39cc5c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:00.695700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6ccd5c04-7366-466e-a19c-d239a39cc5c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:00.696504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:00.698038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 176/363] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:00.698833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:00.699541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:00.699541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:00.702128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:00.714507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:00.721731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:00.723142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:00.817069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 177/364] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:00.818721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.854487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 188/365] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:00.856351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:00.898541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:00.956278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624d95d-876d-4427-a24d-74a080c20e0d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:00.985603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8372ee-f45a-4883-b625-7161a85cd208 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.003960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.004920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.005406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.005866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.006789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:01.007295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.009143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.012392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.017220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 178/366] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.018320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.029624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.030078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.030434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.031343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.036167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:01.036167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.036167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.047345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 189/367] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.050340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.152625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f38436-b44e-4aea-862f-004fa8750e22 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.190928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.191943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.192448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.192987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.194452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.196098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.198445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 179/368] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.204211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.207807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.220203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e705-c0a6-46f7-93c7-b36704d5cfac None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.261685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.262219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.262497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.262768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.293370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.336761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af47005-805a-498c-a4ce-1232e45799e7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.340085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.341581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.380477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.381114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.381444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.382018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.383158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.384655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.386269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 180/369] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.386979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.446607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.456935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 190/370] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:01.461425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.509737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df613-a4a7-4772-8d0a-acb43c16f24c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.548908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.549607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.549885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.550202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.553017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.563370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:01.568396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:01.569258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:01.570050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:01.570980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc8838-8995-4546-8cd5-605fc02b4605 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.571052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 181/371] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:01.571052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.611107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.611723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.612002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.612315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.613049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:01.613356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.614814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.620770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 191/372] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.621569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.682613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723f175-4192-457c-bb86-1d742b6fdfdf None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.713276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.719385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.720039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.720409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.720684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.721902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.723403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.726640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 182/373] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.730757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.797394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c4c80-924f-4c06-b8dc-2f8c13e64cb7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.859401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:01.867566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.868206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.868492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.868768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.869443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:01.869748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.871144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.891761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 192/374] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:01.901250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:01.919383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671b1bf-9bf1-4f67-b46f-859ed8f1f6ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:01.961316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:01.962771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:01.963058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:01.963386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:01.966734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:01.966734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:01.968655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 183/375] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:01.980683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:02.095604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af94b-6780-4cab-aefe-80e53ffac735 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:02.132576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:02.137385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:02.138214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:02.138670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:02.139175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:02.140117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:02.140873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:02.142855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:02.149856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 184/376] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:02.166393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df144fe7-da0e-4eb1-9927-bc5e5b27d4d9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:02.201895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:02.227774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:02.227774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:02.227774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:02.227774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:02.330654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:02.332153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:02.446042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 193/377] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:02.451414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:02.591566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c95cf-7da0-4e64-9411-1f8a22cd8074 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:02.627628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de51dc-f6d6-4e92-b479-a327fd6383d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:02.628311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de51dc-f6d6-4e92-b479-a327fd6383d0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:02.628589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de51dc-f6d6-4e92-b479-a327fd6383d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:02.628860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de51dc-f6d6-4e92-b479-a327fd6383d0 None admin] PATH_INFO: `/v3/projects/0c46f28c367444249b347e64385a501e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:02.630055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de51dc-f6d6-4e92-b479-a327fd6383d0 None admin] RBAC: Authorizing `identity:get_project(project_id=0c46f28c367444249b347e64385a501e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:02.631565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de51dc-f6d6-4e92-b479-a327fd6383d0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:02.633592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 185/378] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:02 2022] GET /identity/v3/projects/0c46f28c367444249b347e64385a501e => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:02.638856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:02.664483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:02.775531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e950a-9b1c-4e84-a851-854fc0e04d3a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:02.791735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba01e82-ee36-44db-817a-a2d926e7c4f9 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:02.792257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba01e82-ee36-44db-817a-a2d926e7c4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:02.792386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba01e82-ee36-44db-817a-a2d926e7c4f9 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:02.793625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 186/379] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:02.799286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:02.799677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:02.800052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:02.824582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:02.825792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:02.836645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:02.837282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:02.837560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:02.837831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] PATH_INFO: `/v3/groups/913674f1bc8b409d935106d41ba6b49b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:02.848221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] RBAC: Authorizing `identity:get_group(group_id=913674f1bc8b409d935106d41ba6b49b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:02.849612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:02.851277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 194/380] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:02 2022] GET /identity/v3/groups/913674f1bc8b409d935106d41ba6b49b => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:02.860847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:02.940760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:02.959803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 187/381] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:03.011277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:03.040091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3857584-236c-4818-b48d-e01f2dd319b0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:03.084238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:03.084853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:03.085131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:03.085403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:03.086124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] RBAC: Adding query filter params (group.id=913674f1bc8b409d935106d41ba6b49b, role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=0c46f28c367444249b347e64385a501e) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:03.086428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:03.087880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:03.096613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 195/382] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:02 2022] GET /identity/v3/role_assignments?group.id=913674f1bc8b409d935106d41ba6b49b&scope.project.id=0c46f28c367444249b347e64385a501e&role.id=c63cf2d08b334ece887cbd2702835968 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:03.277608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:03.411681 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e98f0-c8f8-4422-b68c-176d65604d55 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:03.451773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:03.452756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:03.453178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:03.453616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:03.464566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:03.470176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:03.471234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:03.472267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:03.474528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 188/383] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:03.479855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:03.626679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadcbbc-47c5-4a89-a7a7-9e43de170534 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:03.667356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:03.667971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:03.668275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:03.668550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:03.669222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:03.669530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:03.670977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:03.680629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 196/384] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:27:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:03.685782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:03.770148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:03.817449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c37a67-e84d-4dc6-a384-8d4af107ba5d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:03.854721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3afc-bdec-4224-baec-59494c5d017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:03.855360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3afc-bdec-4224-baec-59494c5d017c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:03.855679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3afc-bdec-4224-baec-59494c5d017c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:03.855907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3afc-bdec-4224-baec-59494c5d017c None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:03.857058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb3afc-bdec-4224-baec-59494c5d017c None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:03.858527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb3afc-bdec-4224-baec-59494c5d017c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:03.860538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 189/385] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:03 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:03.866033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:03.880650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:03.885836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab08f74-c9ac-4d81-afe7-07cc58c1a523 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:03.886118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab08f74-c9ac-4d81-afe7-07cc58c1a523 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:03.886391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab08f74-c9ac-4d81-afe7-07cc58c1a523 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:03.887951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 190/386] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:03.902717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:03.902995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:03.903318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:03.928330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:03.929910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:04.007409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 191/387] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:04.012212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8076377-bc32-4bca-95ef-a8bf80caf509 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:04.045636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:04.046312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:04.046595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:04.046866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:04.047996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:04.049583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:04.051566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 197/388] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:03 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:04.056889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:04.146699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:04.201872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c439fd2-d55c-4acf-b1b2-4998d69ac7d8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:04.252200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:04.252891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:04.253169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:04.253438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/groups/73b79906f98e4a0d90e139a8eefe9848/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:04.255414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a08919df664745e6bf377a7cdfd3cb8e, group_id=73b79906f98e4a0d90e139a8eefe9848, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:04.256874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:04.382489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 192/389] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Sep 7 14:27:04 2022] PUT /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/groups/73b79906f98e4a0d90e139a8eefe9848/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:04.434436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:04.554540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:04.580563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022cd2-3c1e-407b-b856-a38e00ccce09 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:04.619171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447afe77-bec7-40b5-9ec7-f9697257fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:04.620236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afe77-bec7-40b5-9ec7-f9697257fbba None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:04.620739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afe77-bec7-40b5-9ec7-f9697257fbba None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:04.621231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afe77-bec7-40b5-9ec7-f9697257fbba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:04.637910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447afe77-bec7-40b5-9ec7-f9697257fbba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-447afe77-bec7-40b5-9ec7-f9697257fbba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:04.644609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:04.645526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:04.646355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:04.649056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 198/390] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:04.655230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:04.772913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3a3ae-a521-4353-8835-aabcd15db4b0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:04.790147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d04fb-8cd8-453c-b4c2-e5fc3afe0478 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:04.790473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d04fb-8cd8-453c-b4c2-e5fc3afe0478 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:04.792322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d04fb-8cd8-453c-b4c2-e5fc3afe0478 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:04.792453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 199/391] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:04.797399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:04.805381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:04.805721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:04.806006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:04.829437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:04.830710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:04.831168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:04.831615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:04.832517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:04.832651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:04.833117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:04.834636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:04.835277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:04.851135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 193/392] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:04.858047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:04.922248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 200/393] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:05.007697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece9869-fd3f-4ff2-9c04-00f87e14f20b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:05.044162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:05.044865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.045149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.045422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.047121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:05.048636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:05.050510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 194/394] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:04 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:05.056120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.161851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:05.205399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a52ecd-8f3d-43a6-9c67-d6b1e99878eb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:05.252601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff01a1-3b83-4ef6-b1fa-a5891c573dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:05.253240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff01a1-3b83-4ef6-b1fa-a5891c573dca None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.253520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff01a1-3b83-4ef6-b1fa-a5891c573dca None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.253797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff01a1-3b83-4ef6-b1fa-a5891c573dca None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.254939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff01a1-3b83-4ef6-b1fa-a5891c573dca None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:05.256428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff01a1-3b83-4ef6-b1fa-a5891c573dca None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:05.258345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 201/395] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:05 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:05.263986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.267833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b05a0-1d36-4e31-8066-85116ece3a8f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.268120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b05a0-1d36-4e31-8066-85116ece3a8f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.271655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b05a0-1d36-4e31-8066-85116ece3a8f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.271655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 202/396] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:05.275727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.276007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.276308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.307727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:05.307727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:05.329450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:05.388700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 203/397] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:05.390762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.398550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2f3fb-b5cd-4355-9d15-9516ba2e84bc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:05.451666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:05.453531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.453851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.454123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/groups/73b79906f98e4a0d90e139a8eefe9848/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.458021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] RBAC: Authorizing `identity:create_grant(project_id=a08919df664745e6bf377a7cdfd3cb8e, group_id=73b79906f98e4a0d90e139a8eefe9848, role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:05.462551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:05.476907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:05.516512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85c58-2dcf-40d3-b973-117bc191e0a2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:05.558681 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 195/398] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Sep 7 14:27:05 2022] PUT /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e/groups/73b79906f98e4a0d90e139a8eefe9848/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:05.559579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:05.560502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.560955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.561420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.573984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:05.580009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:05.581142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:05.582156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:05.588306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 204/399] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:05.590895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.709350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.739540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576d540-43ba-4f4f-b025-94b210c0b245 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:05.741574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:05.810174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:05.810727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.811077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.811355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.812782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:05.813088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:05.815428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:05.833801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 196/400] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:27:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:05.842047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.853753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce1d3-2e9b-410e-a0c1-134c3ff94607 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:05.878718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:05.889730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:05.890624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:05.890906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:05.891175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:05.902896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:05.908463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:05.909724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:05.910790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 205/401] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:05.915345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:05.969814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9fb7-26db-4f1b-a68d-427715716bbf None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.012828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec1c74-1803-4d11-9d93-eb4f55b5851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.013471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec1c74-1803-4d11-9d93-eb4f55b5851b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.013743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec1c74-1803-4d11-9d93-eb4f55b5851b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.014017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec1c74-1803-4d11-9d93-eb4f55b5851b None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.015946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec1c74-1803-4d11-9d93-eb4f55b5851b None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.017429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec1c74-1803-4d11-9d93-eb4f55b5851b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.019004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 197/402] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:05 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:06.020331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e119d96-6b08-4f8c-81f1-52bf03744230 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.020615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e119d96-6b08-4f8c-81f1-52bf03744230 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.020893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e119d96-6b08-4f8c-81f1-52bf03744230 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.022246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 198/403] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:06.027105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e119d96-6b08-4f8c-81f1-52bf03744230 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.057282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d4014-85a9-4ff3-82c7-cccde71ef93f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.065235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:06.097452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.098449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.098735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.099091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.099661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:06.099959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.101853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8285a-a850-4d43-96bd-d76783d87c28 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.115650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 206/404] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:06.116961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.117317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.117590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.136702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:06.138065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:06.145299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e119d96-6b08-4f8c-81f1-52bf03744230 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.182235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:06.184633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.185502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.185962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.187032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.188406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.190133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.192353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 199/405] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:06 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:06.193265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.212833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 207/406] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:06.213824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.302352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:06.306251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b61e5b-4597-4a6c-9d34-3993779c59ee None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.320653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38704d1-1b61-4188-bdaf-05dee02d076b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.343621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.344602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.345084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.345595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.347218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.348981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.351157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 200/407] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:06 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:06.359583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.361906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.362582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.362861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.363356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/groups/73b79906f98e4a0d90e139a8eefe9848/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.365451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] RBAC: Authorizing `identity:create_grant(project_id=994dc5a0b7fe4fdebc1ce89413258472, group_id=73b79906f98e4a0d90e139a8eefe9848, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.367700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.470177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945e57-6a24-4ebb-b995-9602c386cb1a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.497192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 208/408] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Sep 7 14:27:06 2022] PUT /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/groups/73b79906f98e4a0d90e139a8eefe9848/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:06.507968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.508996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.509609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.510132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.511506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.513176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.515178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 201/409] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:06 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:06.521477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.524657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.565935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:06.592587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:06.652984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca604827-0eb5-4a8f-9a74-dff95b8283aa None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.705006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.705676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.705955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.706227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.715483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8ed0e-209d-42d4-8419-c8a1a058bc54 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.737217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:06.742340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:06.743301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:06.744105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:06.752706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 202/410] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:06.756360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.757056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.757333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.757603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/groups/73b79906f98e4a0d90e139a8eefe9848/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.758957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:06.760411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] RBAC: Authorizing `identity:create_grant(project_id=994dc5a0b7fe4fdebc1ce89413258472, group_id=73b79906f98e4a0d90e139a8eefe9848, role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.762071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.854663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 209/411] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Sep 7 14:27:06 2022] PUT /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472/groups/73b79906f98e4a0d90e139a8eefe9848/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:06.880719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:06.886886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a7fe-8298-4d26-b9b4-32455807cac1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:06.924848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:06.925472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:06.925750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:06.926024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:06.926693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:06.926998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:06.928383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:06.938123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 203/412] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:06.953317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:07.098184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:07.109198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87ed26-aad4-48fa-af26-d137233494ae None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:07.161426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c164dba-40f1-4e58-a02b-2a380e05cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:07.162069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c164dba-40f1-4e58-a02b-2a380e05cb40 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:07.162346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c164dba-40f1-4e58-a02b-2a380e05cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:07.162613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c164dba-40f1-4e58-a02b-2a380e05cb40 None admin] PATH_INFO: `/v3/groups/913674f1bc8b409d935106d41ba6b49b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:07.163969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c164dba-40f1-4e58-a02b-2a380e05cb40 None admin] RBAC: Authorizing `identity:get_group(group_id=913674f1bc8b409d935106d41ba6b49b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:07.166206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c164dba-40f1-4e58-a02b-2a380e05cb40 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:07.168122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 210/413] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:06 2022] GET /identity/v3/groups/913674f1bc8b409d935106d41ba6b49b => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:07.174118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:07.181273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b80b1-c602-44e0-8e97-4cf88ea702da None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:07.181570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b80b1-c602-44e0-8e97-4cf88ea702da None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:07.181905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b80b1-c602-44e0-8e97-4cf88ea702da None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:07.183563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 211/414] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:07.189052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:07.189338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:07.189611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:07.208571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:07.209889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:07.294063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 212/415] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:07.344012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cedc-0d38-47d7-91db-22014ae39af0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:07.398211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:07.398787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:07.399067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:07.399340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] PATH_INFO: `/v3/projects/0c46f28c367444249b347e64385a501e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:07.404836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=0c46f28c367444249b347e64385a501e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:07.406340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:07.408358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 204/416] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:07 2022] GET /identity/v3/projects/0c46f28c367444249b347e64385a501e => generated 344 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:07.413750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:07.543365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:07.550740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cda46-a66f-469f-ad22-a72a1a0f75ed None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:07.594634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:07.595293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:07.595567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:07.595833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] PATH_INFO: `/v3/projects/0c46f28c367444249b347e64385a501e/groups/913674f1bc8b409d935106d41ba6b49b/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:07.597782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c46f28c367444249b347e64385a501e, group_id=913674f1bc8b409d935106d41ba6b49b, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:07.599217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:07.681610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:07.697539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 213/417] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Sep 7 14:27:07 2022] PUT /identity/v3/projects/0c46f28c367444249b347e64385a501e/groups/913674f1bc8b409d935106d41ba6b49b/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:07.716698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:07.807666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd4a3-c317-48cc-a9a0-190d0d49c1d1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:07.852906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:07.853562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:07.853840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:07.854107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:07.869703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:07.875851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:07.876767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:07.877565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 205/418] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:07.884498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:08.009116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee05136-6b9d-4165-94c9-fa86db3266d7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:08.052426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:08.053101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:08.053386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:08.053653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:08.054326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:08.054625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:08.055999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d08d7b-3a02-415b-a6a7-7db3c2b69bb1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:08.065236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 214/419] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:27:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:08.071163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:08.165193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:08.183176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370134-59a3-4e3e-ab39-39f246d235ee None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:08.225258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:08.225802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:08.226081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:08.226352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:08.299984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:08.301483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:08.306741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613e2c-7136-4cae-a426-6809bdb4b38c None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:08.307226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613e2c-7136-4cae-a426-6809bdb4b38c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:08.307311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613e2c-7136-4cae-a426-6809bdb4b38c None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:08.308960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 215/420] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:08.314688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:08.328259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:08.328259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:08.337873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:08.339333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:08.354166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:08.420713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 216/421] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:08.454187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:08.480935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 206/422] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:08.486683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:08.626785 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70ff7b-97b8-4150-8fc0-64a65f17ee76 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:08.664454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:08.665092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:08.665368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:08.665635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:08.666759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:08.668245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:08.670173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 217/423] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:08 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:08.676211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:08.782603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42149d8-1517-4824-bbd2-1fc27c44ed6a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:08.837901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:08.838541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:08.838818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:08.839088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:08.842078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:08.843613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:08.845227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 207/424] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:08 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:08.851162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:08.877775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:08.913719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:08.990477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:09.037537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc812c29-f43a-4229-8175-5f7c16bbdd88 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:09.080782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafadbbe-665f-471c-a495-e6057937c666 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:09.101693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:09.102361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.102705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.102973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.103753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:09.104064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:09.105537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd746-b8e0-4626-91e9-b8086041184c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:09.112817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 218/425] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:08 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:09.166083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e66702-822c-4a8f-90cb-4bc8225f61cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:09.166741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e66702-822c-4a8f-90cb-4bc8225f61cf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.167019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e66702-822c-4a8f-90cb-4bc8225f61cf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.167286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e66702-822c-4a8f-90cb-4bc8225f61cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.188139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e66702-822c-4a8f-90cb-4bc8225f61cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b8e66702-822c-4a8f-90cb-4bc8225f61cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:09.193101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:09.193990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:09.194785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:09.195389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c39d3-c00d-46cf-b0f5-550e57a1e8ef None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.195389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c39d3-c00d-46cf-b0f5-550e57a1e8ef None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.195389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c39d3-c00d-46cf-b0f5-550e57a1e8ef None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.195569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 208/426] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:09.195633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 219/427] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:09.201190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.201448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c39d3-c00d-46cf-b0f5-550e57a1e8ef None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:09.201614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.201780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.228704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:09.229929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:09.314386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 209/428] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:09.380334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c39d3-c00d-46cf-b0f5-550e57a1e8ef None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:09.388503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:09.431451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:09.432065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.432430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.432703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.434349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:09.434653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:09.436034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5abb02-f59c-447d-bfee-5267c04a757a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:09.448451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 220/429] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:09.451628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:09.480107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8651f-39c3-4669-b35b-66b36a414e6e None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.480427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8651f-39c3-4669-b35b-66b36a414e6e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.480679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8651f-39c3-4669-b35b-66b36a414e6e None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.482009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 221/430] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:09.487184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.487727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.488245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.507091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:09.508617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:09.508848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:09.584205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a738f4-557e-4139-bcf3-69f0789b9e02 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:09.599384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 222/431] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:09.626851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:09.627813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.628326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.628816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.652652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:09.713373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:09.715150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:09.746388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:09.770962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:09.781712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e269501-c2fb-48f5-8017-189e82fbcdb6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:09.823245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:09.824456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:09.825238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:09.825732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:09.842507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:09.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:09.849798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 14:27:09.850613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:09.853803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 223/433] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:09.853916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 210/433] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:09.855978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:09.856282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:09.980716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7c88d-6962-4679-b898-f8228d96e267 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:09.984689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f227-d4f9-4cbc-b049-44c2c764a51d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.018853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.019490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.019763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.020028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] PATH_INFO: `/v3/projects/a08919df664745e6bf377a7cdfd3cb8e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.020771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.021193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] RBAC: Authorizing `identity:get_project(project_id=a08919df664745e6bf377a7cdfd3cb8e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.022613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.025489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 211/434] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:09 2022] GET /identity/v3/projects/a08919df664745e6bf377a7cdfd3cb8e => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:10.025604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.029281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.029893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.034853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.034853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.034853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] RBAC: Adding query filter params (name=member) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:10.034853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.034853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.042063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.043931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 224/435] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:27:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:10.048175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.145914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ef3d4-2cd6-4a43-a89f-9d4f6d053228 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.159022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb4327-83e2-4fe5-9f85-cfa24c35749f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.183608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.184269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.184560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.184828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.193389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.194011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.194286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.194551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.195655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.196394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:10.197124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.204353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 225/436] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:10.204353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:10.204573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:10.205376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 212/437] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:10.206140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.269409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.284336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.313676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f4a1b-e18e-49aa-b436-3a4a9996dd60 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.314936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530dd33-7d85-4b0d-a48c-3125cf81ab05 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.349459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.349847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.350077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.350356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.350450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.350628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.350721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.350950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.351660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=a08919df664745e6bf377a7cdfd3cb8e) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:10.351993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.353525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.368426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 213/438] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=a08919df664745e6bf377a7cdfd3cb8e&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:10.368426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.439353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.440978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.499766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b9fd3-e3dc-44c8-9844-c6a729eb7c23 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.516401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.546556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.547688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.548250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.548755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.549650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:10.550172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.551970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.556037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.564418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 214/439] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:10.569777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 226/440] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:10.570127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.575495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.716729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930b49-fb1a-424b-990d-3ee84018dfd2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.722850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0140342-8d75-41c0-b334-5c4181240c96 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.765605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.767802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.768083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.769311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.774274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6d6f6-3c00-41fe-9e06-d474a250702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.776020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6d6f6-3c00-41fe-9e06-d474a250702a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.776357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6d6f6-3c00-41fe-9e06-d474a250702a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.776636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6d6f6-3c00-41fe-9e06-d474a250702a None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.781439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6d6f6-3c00-41fe-9e06-d474a250702a None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.783587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6d6f6-3c00-41fe-9e06-d474a250702a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.785712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 227/441] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:10.787052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d22e4-0823-4962-9c92-1e2b5f8e4ce7 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.787334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d22e4-0823-4962-9c92-1e2b5f8e4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.787605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d22e4-0823-4962-9c92-1e2b5f8e4ce7 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.792770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 228/442] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:10.793578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d22e4-0823-4962-9c92-1e2b5f8e4ce7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.834257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.863404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:10.866708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.868564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.921624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d22e4-0823-4962-9c92-1e2b5f8e4ce7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:10.959076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88f593-6637-4467-9de6-8f4b86719148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:10.959711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88f593-6637-4467-9de6-8f4b86719148 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.959989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88f593-6637-4467-9de6-8f4b86719148 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.960284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88f593-6637-4467-9de6-8f4b86719148 None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.961386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88f593-6637-4467-9de6-8f4b86719148 None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:10.962822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88f593-6637-4467-9de6-8f4b86719148 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:10.964758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 229/443] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:10.966060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:10.966337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:10.966609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:10.984809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 215/444] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:10.985650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:10.986423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:10.987727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:11.052878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 230/445] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:11.053694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:11.079016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:11.086325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:11.090604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a4dfd-1ca4-467c-b15f-5892260bc067 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.125920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.126522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.126796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.127065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.127779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:11.128086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:11.129580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:11.135571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 216/446] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:10 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:11.157025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01d79c-ea01-49e0-9886-0694f0f6992e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.195431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.196421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.196838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.197247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] PATH_INFO: `/v3/projects/994dc5a0b7fe4fdebc1ce89413258472` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.198633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] RBAC: Authorizing `identity:get_project(project_id=994dc5a0b7fe4fdebc1ce89413258472)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:11.200366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:11.202263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 231/447] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/projects/994dc5a0b7fe4fdebc1ce89413258472 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:11.208352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:11.320314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bc1a5-5d72-42b4-b581-fe5d2ecf8e6f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.323482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:11.360577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.361215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.361641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.361925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] PATH_INFO: `/v3/groups/73b79906f98e4a0d90e139a8eefe9848` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.363072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] RBAC: Authorizing `identity:get_group(group_id=73b79906f98e4a0d90e139a8eefe9848)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:11.364540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:11.371165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 217/448] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/groups/73b79906f98e4a0d90e139a8eefe9848 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:11.372523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:11.409399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:11.434464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:11.442585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9a04-81dd-4393-a9dd-9a9d0d0552b7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.480251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.480479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.480699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.480867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.495126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:11.495561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348bdc-3054-4bda-b00d-a2b118f746ab None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:11.500067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:11.501041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:11.502194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 232/449] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:11.507046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:11.536402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.537013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.537289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.537558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.538278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] RBAC: Adding query filter params (group.id=73b79906f98e4a0d90e139a8eefe9848, role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=994dc5a0b7fe4fdebc1ce89413258472) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:11.538584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:11.540003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:11.545980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 218/450] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/role_assignments?group.id=73b79906f98e4a0d90e139a8eefe9848&scope.project.id=994dc5a0b7fe4fdebc1ce89413258472&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:11.613214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88075ce3-03b5-4a5b-ae95-c637f1456459 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.648604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.649219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.649499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.649781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.650460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:11.650766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:11.652182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:11.661957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 233/451] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:11.667712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:11.764030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:11.771389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336873a9-905c-4087-a870-9e0093d4579a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:11.805271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:11.805879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:11.806158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:11.806428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:11.874781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:11.876183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:11.966682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:11.974049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 219/452] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:11.979740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:12.000804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:12.079135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e601b7a-d2b3-4fdf-a10e-c3025776ca9a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:12.111087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:12.111682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:12.111955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:12.112245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] PATH_INFO: `/v3/projects/0c46f28c367444249b347e64385a501e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:12.113332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] RBAC: Authorizing `identity:get_project(project_id=0c46f28c367444249b347e64385a501e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:12.114764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:12.116373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 234/453] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:27:11 2022] GET /identity/v3/projects/0c46f28c367444249b347e64385a501e => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:12.121973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:12.216318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a6fcc-1ef2-4bc1-8f0a-0be19cc5d664 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:12.248359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a50417-4471-48ef-8a24-7e86851b00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:12.249528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a50417-4471-48ef-8a24-7e86851b00af None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:12.249804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a50417-4471-48ef-8a24-7e86851b00af None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:12.250074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a50417-4471-48ef-8a24-7e86851b00af None admin] PATH_INFO: `/v3/groups/913674f1bc8b409d935106d41ba6b49b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:12.251153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a50417-4471-48ef-8a24-7e86851b00af None admin] RBAC: Authorizing `identity:get_group(group_id=913674f1bc8b409d935106d41ba6b49b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:12.252621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a50417-4471-48ef-8a24-7e86851b00af None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:12.254142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 220/454] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Sep 7 14:27:12 2022] GET /identity/v3/groups/913674f1bc8b409d935106d41ba6b49b => generated 229 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:12.259755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:12.343491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:12.359180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064926c-4af2-4578-a550-98b19d6a4998 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:12.392970 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:12.393553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:12.393830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:12.394100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:12.394813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] RBAC: Adding query filter params (group.id=913674f1bc8b409d935106d41ba6b49b, role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=0c46f28c367444249b347e64385a501e) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:12.395119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:12.396700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f64aca-ea0d-44ea-b89a-482269eeabca None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:12.402202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 235/455] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Sep 7 14:27:12 2022] GET /identity/v3/role_assignments?group.id=913674f1bc8b409d935106d41ba6b49b&scope.project.id=0c46f28c367444249b347e64385a501e&role.id=c63cf2d08b334ece887cbd2702835968 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:14.999916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31dadb-65f1-4e09-958e-4e36d64d7b42 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.000494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31dadb-65f1-4e09-958e-4e36d64d7b42 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.000494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31dadb-65f1-4e09-958e-4e36d64d7b42 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.001810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 221/456] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:15.007634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077608b-26df-4181-af81-a1bac16de005 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.008095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077608b-26df-4181-af81-a1bac16de005 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.008217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077608b-26df-4181-af81-a1bac16de005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.034642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-c077608b-26df-4181-af81-a1bac16de005 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:15.035852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c077608b-26df-4181-af81-a1bac16de005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:15.138462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 236/457] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:15.278853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf3d3-b1c8-40b0-9390-57bd4943ddd9 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.279610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf3d3-b1c8-40b0-9390-57bd4943ddd9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.280133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf3d3-b1c8-40b0-9390-57bd4943ddd9 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.282029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 222/458] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:15.286918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf088e-3359-45e7-ad60-fa45e31228b7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.287152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.287519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.287863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf088e-3359-45e7-ad60-fa45e31228b7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.288039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.288537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf088e-3359-45e7-ad60-fa45e31228b7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.290205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 223/459] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:15.296267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.296835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.297321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.314528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:15.316174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:15.337895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:15.339336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:15.447053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 237/460] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:15.480540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:15.486656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 224/461] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:15.498407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:15.509352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:15.605711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0a0ac-a3b3-49f4-9784-5117a5aeb8be None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:15.647186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:15.647820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.648095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.648396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.659467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:15.660987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:15.685346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:15.686400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:15.687254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:15.687254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:15.687254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:15.687254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 238/462] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:15.692974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:15.783845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:15.858263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:15.862929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f7eb8-2987-4dc8-ad15-7ddb2689e70d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:15.907000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410908b8-2e65-435f-8797-84a121244a06 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:15.934263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:15.935136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.935563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.936554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.937346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:15.937795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:15.939292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:15.945282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:15.946224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:15.946695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:15.947157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:15.951600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 225/463] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:15.952698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:15.960253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:15.961868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:15.992377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:15.993379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:15.994228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:15.994228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:15.994228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:15.997058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 239/464] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:15.999170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:16.024878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:16.080273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a537eb-8268-4fbd-ad29-9c1c22ecd803 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.124529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.125088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.126746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.126746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.126911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35553646-6259-4f45-bd1b-c405d13c87d9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.142533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.144099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:16.145367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:16.159824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:16.160869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:16.161705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:16.161705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:16.161705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:16.163151 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 226/465] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:16.164055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:16.173269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca50d79-e2fb-4c15-9f81-0a1fc5021677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.174140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca50d79-e2fb-4c15-9f81-0a1fc5021677 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.174669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca50d79-e2fb-4c15-9f81-0a1fc5021677 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.174937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca50d79-e2fb-4c15-9f81-0a1fc5021677 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.175581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca50d79-e2fb-4c15-9f81-0a1fc5021677 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.177059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca50d79-e2fb-4c15-9f81-0a1fc5021677 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:16.263895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0fdea-76e6-4612-994e-9891bc2b2704 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.267065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:16.284746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 240/466] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:27:15 2022] POST /identity/v3/users => generated 246 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:27:16.289689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f5fcf-204a-48ab-9306-11f269bf5ace None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.289689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f5fcf-204a-48ab-9306-11f269bf5ace None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.289689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f5fcf-204a-48ab-9306-11f269bf5ace None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.289689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 241/467] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:16.289689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f5fcf-204a-48ab-9306-11f269bf5ace None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:16.315541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.316706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.316994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.317273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.317958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:16.318264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.319696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f5423-6771-47e6-80af-58558a5a2250 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:16.337470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 227/468] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:16.337470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.337470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.337470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.362649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:16.364063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:16.432054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f5fcf-204a-48ab-9306-11f269bf5ace None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.462691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 228/469] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:16.462941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:16.478564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.479745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.480569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.481146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.482142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:16.482667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.484535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:16.505198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 242/470] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:16.508443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:16.544875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:16.583802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:16.593932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0a29d-105d-4388-88f5-2bd2ab0e8d0c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.638707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf70f8-2f0c-4b44-890b-4abbdbde0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.639887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d07c8d-c617-4a2b-aed1-9832232d0541 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.640085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf70f8-2f0c-4b44-890b-4abbdbde0b9f None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.640839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf70f8-2f0c-4b44-890b-4abbdbde0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.641258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf70f8-2f0c-4b44-890b-4abbdbde0b9f None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.641922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf70f8-2f0c-4b44-890b-4abbdbde0b9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.643635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf70f8-2f0c-4b44-890b-4abbdbde0b9f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:16.689403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.690350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.690647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.690905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.691559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.693037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:16.736790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 229/471] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:27:16 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:27:16.738434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3cc25-3727-4987-b67a-199ff5c73afe None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.739027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3cc25-3727-4987-b67a-199ff5c73afe None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.739517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3cc25-3727-4987-b67a-199ff5c73afe None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.741336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 230/472] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:16.750460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.750788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.751010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.772607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:16.781845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:16.808939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 243/473] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:27:16 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:27:16.811983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:16.872115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 231/474] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:16.926717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:16.945639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a684d87-bfcb-4adb-97e4-33ec4afe0376 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:16.983411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:16.984052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:16.984360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:16.984626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:16.996025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:16.997606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:17.016056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:17.017050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:17.017887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:17.017887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:17.017887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:17.017887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 244/475] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:17.025336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:17.113812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:17.133025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9eb9-fadc-4551-9403-5735846a2563 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:17.169650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:17.170332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:17.170611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:17.170877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:17.171524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:17.171823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:17.173280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:17.185566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 232/476] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:17.191686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:17.197913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:17.305704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb102-18bd-40b3-95f4-1c64f8ce6a9e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:17.321185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8278fd-ddaf-474f-960b-278eddfd8e62 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:17.356850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:17.358351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:17.358849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:17.359331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:17.360266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:17.369570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:17.410633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:17.411270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:17.416165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:17.416165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:17.416165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:17.416165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:17.426902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.sql.core [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 14:27:17.426902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 14:27:17.426902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [parameters: {'id': 'b24bf2a6d18e43168e5f2ad66f8c528e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 14:27:17.426902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81805) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 14:27:17.427274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application conn.query(q) Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result.read() Sep 07 14:27:17.443890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 14:27:17.444904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application next(self.gen) Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application yield resource Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application next(self.gen) Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise self.value Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application session.commit() Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 14:27:17.446189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.session.flush() Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application compat.raise_( Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 14:27:17.447153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 14:27:17.448183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application conn.query(q) Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result.read() Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 14:27:17.449041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [parameters: {'id': 'b24bf2a6d18e43168e5f2ad66f8c528e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:17.449912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 14:27:17.450712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:17.453435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 233/477] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Sep 7 14:27:17 2022] POST /identity/v3/roles => generated 144 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:17.453435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:17.477770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:17.550727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 245/478] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:27:17 2022] POST /identity/v3/users => generated 249 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:27:17.563459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e3d60-6f41-41e5-9ae4-b50111b77847 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:17.604081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b953d-f6d2-47cb-874e-e36669726622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:17.605153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b953d-f6d2-47cb-874e-e36669726622 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:17.605633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b953d-f6d2-47cb-874e-e36669726622 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:17.606119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b953d-f6d2-47cb-874e-e36669726622 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:17.619041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b953d-f6d2-47cb-874e-e36669726622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cc7b953d-f6d2-47cb-874e-e36669726622 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:17.624651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:17.625570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 14:27:17.626357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:17.629101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 234/479] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:17.635488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:17.777434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b04768-ac50-4a8b-9b20-c6f8e6d1e0cb None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:17.815100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:17.815709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:17.815984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:17.816290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:17.816961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:17.817314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:17.818771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b117f-2bfb-4a21-bbe8-5c7cd2a253b4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:17.828219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 246/480] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:19.126853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b32dc-018b-46de-88ca-ababb9566ab1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.127786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b32dc-018b-46de-88ca-ababb9566ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.128252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b32dc-018b-46de-88ca-ababb9566ab1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.130279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 235/481] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:19.136046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030abd2-b452-48cd-924d-4e65a94f9457 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.137045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030abd2-b452-48cd-924d-4e65a94f9457 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.137549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030abd2-b452-48cd-924d-4e65a94f9457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.160365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6030abd2-b452-48cd-924d-4e65a94f9457 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:19.161915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6030abd2-b452-48cd-924d-4e65a94f9457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:19.225033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:19.253963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 247/482] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:19.391151 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777e563-1c0c-40dd-8dc6-e351652c5c79 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.391492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777e563-1c0c-40dd-8dc6-e351652c5c79 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.391688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777e563-1c0c-40dd-8dc6-e351652c5c79 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.393035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 236/483] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:19.398368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.398654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.398925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.419969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:19.421739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:19.513255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 248/484] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:19.605827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777e563-1c0c-40dd-8dc6-e351652c5c79 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:19.689202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:19.739904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777e563-1c0c-40dd-8dc6-e351652c5c79 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:19.788908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:19.789480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.789759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.790030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.796635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:19.808166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:19.809076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:19.809924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:19.812250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 237/485] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:19.814449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:19.933318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991793bb-9c98-4f02-9edc-1763db3e6b53 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:19.944393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2682537-6fc4-47d9-9849-e3895773eb15 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:19.959716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:19.963172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:19.981025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:19.981638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.981911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.982177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.983004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:19.983309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:19.984740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ca94-bbb9-4894-893d-371281a9837a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:19.985929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:19.987290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.987787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.988298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:19.998666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 238/486] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:19.998666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94623256-0a25-42c9-b5d1-d0e9e60d99fc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:19.998666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94623256-0a25-42c9-b5d1-d0e9e60d99fc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:19.998666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94623256-0a25-42c9-b5d1-d0e9e60d99fc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.002235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 239/487] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:20.003339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.009464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.009464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.009627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:20.015617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:20.016536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:20.017320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:20.020429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 249/488] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:20.021439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.028735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:20.030211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:20.109029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 240/489] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:20.109886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.150441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcfd30-c855-489b-93c9-23271ca3611a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.201737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.203194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.203677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.204605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.219306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:20.221738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:20.228864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeae0b6-04a4-436d-9ebe-85a65d9d5508 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:20.243218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:20.244163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:20.245045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:20.245045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:20.245045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:20.250101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 250/490] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:20.253417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.255432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:20.292027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.292706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.292995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.293264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.293939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:20.294248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:20.296352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:20.308072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:20.309479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 241/491] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:20.315101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.434141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24237bd-1934-4fcf-8a99-8aa5d0bbb1e2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.506850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24205d-a510-4bec-a137-7a52cfe286fe None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.510445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.511059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.511343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.511615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.512348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:20.512652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:20.514559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:20.539931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 251/492] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:20.541261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.555650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.556364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.556643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.556913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.569498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:20.570982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:20.586408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:20.587358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:20.588394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:20.588394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:20.588394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:20.588394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 242/493] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:20.588617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.618595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:20.632908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:20.648021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b119210-502f-4d9e-8f49-3321280bb0bf None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.683272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88d037-4464-41b0-ae6f-c893cbc4de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.683901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88d037-4464-41b0-ae6f-c893cbc4de54 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.684253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88d037-4464-41b0-ae6f-c893cbc4de54 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.684473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88d037-4464-41b0-ae6f-c893cbc4de54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.692916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2ec3-ac26-4109-80f0-08339c59d8c4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.694989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88d037-4464-41b0-ae6f-c893cbc4de54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-ae88d037-4464-41b0-ae6f-c893cbc4de54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:20.699780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:20.700820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:20.702181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 252/494] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:20.702824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7009f70-f151-40ca-80b6-a0ded6077d3b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.702824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7009f70-f151-40ca-80b6-a0ded6077d3b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.703819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7009f70-f151-40ca-80b6-a0ded6077d3b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.704474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 253/495] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:20.705344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfea645-005b-481f-96e8-f82e0860301c None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.705623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfea645-005b-481f-96e8-f82e0860301c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.705889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfea645-005b-481f-96e8-f82e0860301c None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.710303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 254/496] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:20.711071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfea645-005b-481f-96e8-f82e0860301c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.731799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.732467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.733289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.733562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.740570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:20.742020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f01acb9e-4bb5-43c9-99b1-f0d38a71a977 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:20.754637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:20.755523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:20.756472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 243/497] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:20.757026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.757262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.757559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.775482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:20.776702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:20.816446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfea645-005b-481f-96e8-f82e0860301c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.843204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 244/498] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:20.844021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.863632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.864453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:20.864682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.864947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.865530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.866313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:20.866480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:20.868302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86632c-9c6a-424e-a220-9e08ce3f336c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:20.882387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 255/499] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:20.884164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.884498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.884748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.902585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:20.908953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:20.910294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:20.953709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee28c6-9e35-4d4c-80e3-6a339b825503 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:20.990917 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 256/500] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:20.991771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:20.995709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:20.996568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:20.997034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:20.997463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:20.998812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:20.999273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.001365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.012461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 245/501] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:21.013513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.109770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.124396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e32a08-3959-4df9-9cf9-1a413b073caf None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.129434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dbd51-c1e4-47a6-af8d-1d16ee34bdbb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.161852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.162501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.162722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.162995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.163722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:21.164025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.165352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.165511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.166265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.166707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.167121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.175150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.176769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.178187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:21.190140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:21.191054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:21.191935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 246/502] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:21.192537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.214845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 257/503] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:27:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:21.214845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.304040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94607e3b-4083-44bd-84e5-9668c9872711 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.320416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237223-667f-4f37-8ad5-01e6d7d605df None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.337426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.338057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.338339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.338610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.349636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.351084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.352879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:21.365833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:21.366776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:21.367710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:21.367710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:21.367710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:21.367710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 247/504] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:21.367710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.379385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.380053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.380355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.380626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.391388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.392886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:21.407794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:21.408717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:21.409579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:21.409579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:21.409579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:21.409579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 258/505] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:21.409792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.456698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.470842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a858322-175a-4d8d-b85d-f47293ce67c2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.503449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.504051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.504355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.504632 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.505352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:21.505653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.507073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.508552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875dca6-723a-4fc1-8caf-eddbaf39d7f3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.541266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.541860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.542132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.542396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.548133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 248/506] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:21.548133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.600104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.609514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.611047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.647432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dba44-6ffb-4609-a881-a6747a14e8bd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.657126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.680248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.680875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.681152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.681422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.692299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:21.697509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:21.698399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:21.699215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 249/507] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:21.699861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.716448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 259/508] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:21.716448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.797456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550c041-e5c3-4e70-9993-03da35fcbd7d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.809597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.810759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe144d24-d347-46bc-8e5e-26b282780f35 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.830492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.831087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.831355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.831620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.832280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:21.832574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.833950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.844824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.845417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.845689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.845951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.846580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:21.846877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.847252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 250/509] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:21.848045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.848279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:21.859581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 260/510] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:21.860364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:21.869649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:21.951866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843b88-a43b-4387-9507-28511e32c986 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.959286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64bfce-aa03-4b12-ab71-475b757d24c9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:21.987240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.987848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.988130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.988429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.992735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:21.993334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:21.993619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:21.993888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:21.994549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:21.994851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:21.996252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.009077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 261/511] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:22.009077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.029599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.055362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.056806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.075472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.108289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757417-bfa2-41e9-9a32-d8f5374bbe51 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.141767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.142385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.142670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.142941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.153157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.155107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.156105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 251/512] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:22.156947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:22.173915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:22.175179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:22.176029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:22.176029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:22.176029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:22.176029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 262/513] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:22.176029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.225740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.255245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3987cd0-fd71-4286-bca9-8079a2cceec0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.276965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689eb824-8cdd-4bba-a1b7-64f7295b5651 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.288989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.289618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.289898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.290165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.297638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.298086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.299094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:22.311111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:22.312524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 14:27:22.313405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:22.313911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.313996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 252/514] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:22.314061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.314061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.314234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.314302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.314302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.315287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.389419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 263/515] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:27:22 2022] POST /identity/v3/users => generated 247 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:27:22.390468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.428994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d54c-76c9-43a0-8cf4-10535c2407e4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.462419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.470307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.471119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.471411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.472639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.486593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.488080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:22.504849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:22.505953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:22.506914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:22.506914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:22.506914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:22.506914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 253/516] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:22.507163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.516403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf9a09-f2bd-4dda-bec7-28b8e75b8005 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.555873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.556540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.556819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.557089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.570127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.571562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:22.586890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:22.587788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:22.590327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:22.590327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:22.590327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:22.590327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 264/517] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:22.590327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.624253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.636276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d803aa-3764-40e1-9406-551d00958276 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.673704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.674314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.674585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.674855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.675490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:22.675785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.677272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.689606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 254/518] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:22.690418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.708234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b8708-2107-4357-81f0-5b381c74e1b7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.736620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.746513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.751327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.751327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.751327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.751327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:22.751327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.751327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.773597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 265/519] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:22.774495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.792828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478447b-813a-46dc-b2ea-ae0d39389a40 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.828837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.829441 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.829721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.830292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.830617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:22.830917 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.832312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.844959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 255/520] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:22.845662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.863567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:22.879516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129dee-8eee-4237-81cb-10690ab119ef None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.917541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:22.918147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:22.918423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:22.918693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:22.919328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:22.919630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:22.921596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:22.937284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 266/521] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:22.937284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:22.963475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9037-4275-452b-99a3-63250e9340e2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:22.969408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.006019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.006654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.006933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.007204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.014350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.016403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:23.028696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:23.030194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:23.031024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 256/522] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:23.031521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.044374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a46c7-c6cc-451d-93a6-7f338fef9dff None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.078041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.078640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.078979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.079198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.094778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.140513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2df77d-9f6c-4119-8d4e-d4e4dae479d6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.148303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.149808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.176070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.176746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.177025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.177491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.186279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.187736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:23.199417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:23.200424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 257/523] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:23.201922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.229022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.263733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 267/524] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:23.263733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.312124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.317677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca3856-444c-4db6-84bd-4cd47390c266 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.356895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.358243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.358523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.358794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.366788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.368359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.368955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378862a5-e817-4ebb-9cf3-2d505e36a2f5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:23.380269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:23.381112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 258/525] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:23.383168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.410615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.411217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.411494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.411772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.412504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:23.412807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.414262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.432923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 268/526] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:23.433745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.481753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.496991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e201e-c208-475d-bbcd-e8743ca8c07e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.533322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.534141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.534251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.534467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.535188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:23.535492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.536946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.540208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6eaaa-a6ea-4199-9fb0-4e7944d645da None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.547671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.554680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 259/527] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:23.555354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.579864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.580545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.580823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.581093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.593117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.594611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:23.609637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:23.610540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:23.616875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:23.616875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:23.616875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:23.616875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 269/528] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:23.616875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.666498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff9a96-5990-46fa-ba55-31c187137839 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.703453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.704072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.704392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.704696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.705386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:23.705690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.707118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.724334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cefdc8-5cc1-40d2-a41c-f321a4a36d3d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.729754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 260/529] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:23.729754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.729754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.765036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.765909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.766190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.766461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.767182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=0cb8a6f42b6e482eb109be9d2f40e64f) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:23.767495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.768984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.774439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 270/530] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/role_assignments?user.id=0cb8a6f42b6e482eb109be9d2f40e64f&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:23.775215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:23.814450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:23.855554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822180e2-06f4-49c4-b616-1b27d05ff7c3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.906527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.907068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.907349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.907622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.919100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad5d37-82d4-4eb8-8ccc-f8ad21fa0381 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:23.982171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:23.983074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:23.983796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:23.984075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:23.985043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:23.985352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.987588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:23.996643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:23.998722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.000847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:24.009498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 271/531] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:24.009498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.113671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 261/532] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:24.114373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.130008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:24.156110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd157c-c344-4897-b9be-ca266b3da9b1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.193192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.193721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.193998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.194268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.195320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=d83045f9a43b48d7bbcce339f7bfa5a2) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:24.195636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.197187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.204540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 272/533] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/role_assignments?user.id=d83045f9a43b48d7bbcce339f7bfa5a2&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:24.205738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.236912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9638a-6699-4d93-a2b2-15b05f49ee1f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.239473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:24.294442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.295801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.296086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.296380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.312379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.313889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:24.329393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:24.330755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:24.331635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 262/534] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:24.337415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.376117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b529ee-d417-487e-9d2f-2c82e4d9410a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.443915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.444767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.445051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.446210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.467314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.468874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.486262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:24.500776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:24.505045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:24.505913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:24.505913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:24.505913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:24.505913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 273/535] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:24.509802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb718a7-5a8a-426b-b2f2-5784d41d31cb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.513380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.575494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.576203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.576987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.577260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.577962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:24.578262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.582882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.606978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 263/536] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:24.611581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.650170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:24.674731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf3a06-4bb6-4c9f-a2b0-2dc2b62b19e8 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.712453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.712982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.713257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.713528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.714167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:24.714528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.715911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686facf6-8f2d-4bac-8dbf-c7c281663d76 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.729330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 274/537] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:24.730588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e8c16-5cda-41b8-b6ad-128b719b2c39 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.730867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e8c16-5cda-41b8-b6ad-128b719b2c39 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.731135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e8c16-5cda-41b8-b6ad-128b719b2c39 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.737027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 275/538] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:24.737804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e8c16-5cda-41b8-b6ad-128b719b2c39 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:24.758780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd33b7-b6c1-4b26-afae-d7bd9947731a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.806339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:24.808624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.809251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.809530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.809804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.811043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=113ccc244a814d7580d1c9f6319edf04) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:24.811357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.813114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656cc1-7d9b-4442-9e17-dd99bd11c3e7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.820976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 264/539] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/role_assignments?user.id=113ccc244a814d7580d1c9f6319edf04&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:24.822267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.822551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.822823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.862992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:24.864277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:24.885416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e8c16-5cda-41b8-b6ad-128b719b2c39 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:24.944311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:24.944955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:24.945233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:24.945500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:24.954317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:24.956408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:24.984554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:24.985533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 276/540] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:24.986581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:25.022507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 265/541] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:25.137203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c05d-070e-4e96-ae36-be0f5acce1ff None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:25.168530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:25.176466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:25.177077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:25.177354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:25.177627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:25.178327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:25.178639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:25.180079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:25.193163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 277/542] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:25.199145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:25.232353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:25.333230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:25.347229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e577c40-18e5-4bd8-b4c3-f9ca905ee489 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:25.385047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:25.385992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:25.386466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:25.386930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:25.387868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=b50c8a35e28e44bba3a4b4e56057a3fe) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:25.388412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:25.391087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:25.397670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 266/543] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:25 2022] GET /identity/v3/role_assignments?user.id=b50c8a35e28e44bba3a4b4e56057a3fe&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:25.528631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338d9e4-6fa5-45b0-a888-01366f729e8f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:25.581775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:25.582428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:25.582710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:25.582979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:25.596328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:25.601224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:25.602095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:25.602940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 278/544] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:25.608897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:25.743393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:25.772668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916f14e-ac9a-4c75-8807-12b67234e91a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:25.817128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:25.817742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:25.818021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:25.818341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:25.819036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:25.819317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:25.820710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:25.830166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 267/545] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:25.835466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:25.888008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:25.966519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adcfd7-91ed-4ce5-a102-e87cfb0426f4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:26.017888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:26.018526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.018806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.019078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.034036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:26.035500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:26.054265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:26.055276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:26.056110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:26.056110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:26.056110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:26.056110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 279/546] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:26.061654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:26.176443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79375f-76f1-44d4-8546-c0f33defbe12 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:26.214879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:26.215514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.215793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.216065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.217306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:26.217612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:26.219031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e64cf-41e0-406b-a254-de430957e9e2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:26.233404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 268/547] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:26.241433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:26.317826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c1fc-3fc6-4c3e-a92c-40fd47764353 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.318176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c1fc-3fc6-4c3e-a92c-40fd47764353 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.318384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c1fc-3fc6-4c3e-a92c-40fd47764353 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.319850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 269/548] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:26.325072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.325381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.325623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.338085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:26.353515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:26.355435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:26.377865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:26.387351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e24d7-c42a-45b3-819e-5080aa6c9cc4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:26.436116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 270/549] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:26.456853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:26.457496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.457772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.458049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.469160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:26.470613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:26.490241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:26.491183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:26.492069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 280/550] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:26.498845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:26.632018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b004-99ce-4ad5-a65b-524fe1847051 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:26.681205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:26.686750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:26.687361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.687637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.687907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.688673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:26.688974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:26.694177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:26.722522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 271/551] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Sep 7 14:27:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:26.728093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:26.767520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:26.880586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783d24d-08e7-42a9-8753-2de3202937f4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:26.914513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaab34-bc34-42c5-8889-72db8ae89586 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:26.928346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760a7ec-3add-4017-8d88-191ec3a4b8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:26.929657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760a7ec-3add-4017-8d88-191ec3a4b8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.930396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760a7ec-3add-4017-8d88-191ec3a4b8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.930873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760a7ec-3add-4017-8d88-191ec3a4b8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.959695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa0fdf-4125-4595-998d-f24109c41ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:26.960353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa0fdf-4125-4595-998d-f24109c41ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.960633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa0fdf-4125-4595-998d-f24109c41ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.961162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa0fdf-4125-4595-998d-f24109c41ba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.976522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:26.983568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa0fdf-4125-4595-998d-f24109c41ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-31fa0fdf-4125-4595-998d-f24109c41ba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:26.989195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:26.996870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 272/552] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:26.997765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f856d58-75d3-4bfb-8180-d81548877195 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:26.999452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f856d58-75d3-4bfb-8180-d81548877195 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:26.999452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f856d58-75d3-4bfb-8180-d81548877195 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:26.999452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 273/553] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:26.999452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f856d58-75d3-4bfb-8180-d81548877195 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.041407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760a7ec-3add-4017-8d88-191ec3a4b8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.043430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760a7ec-3add-4017-8d88-191ec3a4b8f3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.064168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.120739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f856d58-75d3-4bfb-8180-d81548877195 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.152230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 281/554] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:27.152230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.152230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.152230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.163542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.164331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.164734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.165166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.165928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:27.166333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.167812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.171034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:27.172272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:27.181775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 274/555] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:27.182820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.223021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.246390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 282/556] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:27.247334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.334303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818df5-8f78-48ff-88f8-c72cf1e032e5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.359709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7ba40-1aa8-4bcc-882c-dc20073cd007 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.366061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.397129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.397852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.398220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.398491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.412114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.418064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.422157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.422807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.423087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.423360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.434650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.436712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:27.448426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:27.449314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:27.450570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:27.450570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:27.450570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:27.450570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 283/557] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:27.452170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:27.453881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:27.456011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:27.456011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:27.456011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:27.456011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 275/558] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:27.460182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.465121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.520592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.647225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab665b-c210-4923-ab70-fcd5e09633b8 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.650501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d0b7d-ef2c-4907-aec2-baff6ff11f05 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.683214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.683822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.684099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.684395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.685035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:27.685148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.685867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.686260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.686540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.686814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.687240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.687454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:27.687821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.689221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.699747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 284/559] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:27.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 276/560] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:27.706268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.707619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.733131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.802364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.812512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4b86c-176d-4c06-9c1e-61dba9c5c0a9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.820196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ddf56-1145-4f56-b3b2-8f209025e378 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:27.852958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.853591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.853866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.854134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.859005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:27.859908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.860503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.860987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.865318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:27.868655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:27.870108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:27.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:27.872040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 285/561] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:27.872748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.873641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7bab9337-b67a-4800-9629-baa7b4bb4c8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:27.889072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:27.890443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:27.891368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:27.895570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 277/562] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:27.896799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4752f-fe87-41d8-acd1-d10315fb3c4b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:27.897084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4752f-fe87-41d8-acd1-d10315fb3c4b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:27.897361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4752f-fe87-41d8-acd1-d10315fb3c4b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:27.898769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 278/563] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:27.899324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4752f-fe87-41d8-acd1-d10315fb3c4b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:27.989197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:27.989726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecd99-9ab5-4233-ba0d-1c9320463b72 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.005293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4752f-fe87-41d8-acd1-d10315fb3c4b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.022705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.023319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.023599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.023863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.031002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.032478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.039992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.040686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.040958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.041219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.041863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:28.042157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.043482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314ad1-65de-4155-b8c4-fbe9fcb6d780 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:28.043588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:28.044462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:28.045923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 286/564] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:28.048963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.052613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 279/565] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:28.053836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5795fc-6a7a-4811-8a23-4a6911843e6e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.054118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5795fc-6a7a-4811-8a23-4a6911843e6e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.054389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5795fc-6a7a-4811-8a23-4a6911843e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.064650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.072095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8e5795fc-6a7a-4811-8a23-4a6911843e6e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:28.073308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8e5795fc-6a7a-4811-8a23-4a6911843e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:28.137845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 280/566] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:28.139286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a8c1-4888-4c82-a768-77ee9b0efb3b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.139569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a8c1-4888-4c82-a768-77ee9b0efb3b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.139839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a8c1-4888-4c82-a768-77ee9b0efb3b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.141353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 281/567] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:28.142033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a8c1-4888-4c82-a768-77ee9b0efb3b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.159016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091851-b908-43e2-a8b1-26e60a071bac None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.193355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.193964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.194243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.194512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.195239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:28.195543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.197005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.213891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 287/568] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:28.214683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.236450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.261633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a8c1-4888-4c82-a768-77ee9b0efb3b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.288778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.299974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.300829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.301302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.301583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.302271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:28.302573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.304425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a45381-4a54-439e-9025-b7e1070e6c94 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.320911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 282/569] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:28.320911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.320911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.320911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.326248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed163c-8559-4472-8555-4a4e3a84ef2e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.359498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.360124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.360428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.360699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.369002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:28.370191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:28.371336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.373339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:28.389969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:28.391173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:28.392096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:28.392096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:28.392096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:28.392096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 288/570] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:28.392971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.448313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 283/571] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:28.449159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.508913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc65e51-5f1e-47e0-9742-06c98b4edbc2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.517191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.532127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.543681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.544334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.545126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.545489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.555640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e2d93-a6c9-4ddf-bd1d-00e44c158cca None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.555973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.557476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:28.572081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:28.573053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:28.573923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:28.573923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:28.573923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:28.573923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 289/572] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:28.574226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.596468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.597076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.597346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.597607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.598310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=256b9d949c8e4734a6bce965a107c060) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:28.598609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.600028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.616536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 284/573] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/role_assignments?user.id=256b9d949c8e4734a6bce965a107c060&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:28.617326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.695585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dd2bf-1cf4-4098-9a04-29b63bdabf1a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.734972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.735614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.735894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.736200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.737024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:28.737335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.739511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.758289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 290/574] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:28.759169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.772788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355df53-d0ac-44af-8386-4bb90bf2f3bc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.797074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.817208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:28.819918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.820333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.820617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.820866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.832123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:28.837038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:28.840941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 285/575] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:28.842331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.873901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab13e7-a0d4-4fc5-ba1e-5c1ef66e77d3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:28.912280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:28.913303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:28.913829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:28.914295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:28.915084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:28.915382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:28.916768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:28.935049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 291/576] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:28.935049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:28.961780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069a33b-1835-4461-b164-bff0ddbcb3f9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.000742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.001795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.002073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.002435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.012789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:29.018981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:29.019885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:29.020755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 286/577] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:29.021433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.055255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.062958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8a22-3c9f-4ba8-99a8-59f8b56f9941 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.077928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.099893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.100546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.100819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.101088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.109786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.111246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:29.122516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:29.123459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 292/578] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:29.124416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.136578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8ce1f-ef8b-4044-b0b2-ab7dfffe121f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.172559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.173320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.173657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.173938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.174623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:29.174926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.176304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.185281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 287/579] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:29.186143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.236135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b31ae-f578-4b87-bdf1-88bf99f2ecdf None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.271389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.272111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.272477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.272911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.280630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.282266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.299312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0576ad-a9f6-41c4-96f8-1f8d57cd5557 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:29.301809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:29.302833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 293/580] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:29.308882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.323869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.357170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.378179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.378776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.379067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.380003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:29.380334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.381761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.393098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 288/581] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:29.398103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.414932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd5bb6-71cd-449f-bc2d-d7bd2ff3f150 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.448287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.449190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.449190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.449413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.450101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:29.450402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.452368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.470754 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 294/582] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:29.473142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.506453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccba8b-1bd9-4f91-9d2e-e8f55a01153a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.542963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.543588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.543861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.544128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.553198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.554585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.556018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:29.571012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:29.572168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:29.573262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:29.573262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:29.573262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:29.578842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 289/583] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:29.578842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.590081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57777d6d-0b8c-4e6f-820f-792c09933d2f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.605755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.626261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.626791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.627065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.627334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.628030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:29.628353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.629769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.647481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 295/584] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:29.647481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.694701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238556d-42c2-46c8-bd69-34763053f3b9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.731397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.732033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.732376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.732654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.743302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.744819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.752293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975eb7e-1d3b-49ac-9c15-eee07ed52219 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:29.759738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:29.767075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:29.768284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:29.768284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:29.768284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:29.768284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 290/585] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:29.768284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.794571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.801125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.801873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.802190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.802580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.814424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.816412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:29.831652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:29.832746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:29.833666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:29.833666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:29.833666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:29.833666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 296/586] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:29.833666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.849909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:29.871440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7ea10-fb6f-455a-ad0c-7c9fd57e2898 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.906832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.907532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.907893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.908336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:29.909082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:29.909492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:29.910977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:29.927109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 291/587] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:29.927109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:29.948811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225736-3b8b-4709-8c44-de9e4d810af0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:29.992597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:29.993476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:29.993755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:29.994029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/0cb8a6f42b6e482eb109be9d2f40e64f/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.009638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=0cb8a6f42b6e482eb109be9d2f40e64f, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.011080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.037038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5cb66-b6d9-4dce-8ae9-a47a5cddfa16 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.071064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.071662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.071939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.072241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.072923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:30.073225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.074579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.081344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.086660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 292/588] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:30.087516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.104227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 297/589] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:29 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/0cb8a6f42b6e482eb109be9d2f40e64f/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:30.104227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.137280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.217294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e571b4-b262-4a08-8fc9-5a9fe1e5ecf9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.241185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1a57a-2a9d-4fad-a7da-17c0406b2f95 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.260493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.261480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.261765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.262112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.271282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.272951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:30.286938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:30.287951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:30.289313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 298/590] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:30.289898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.291339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.292202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.292815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.293675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.296177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:30.296690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.298476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.322937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 293/591] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:30.322937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.393616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.410655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.418078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4ae5d-3f83-4d59-aaf5-e6be7189f892 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.437336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15ee5-111e-41f9-844d-5ec3661335ca None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.455535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.456578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.457070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.457559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.465672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.470127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.474130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.474837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.475189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.475537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.476368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:30.476743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.478238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:30.482981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:30.483914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 14:27:30.484999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:30.487230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 299/592] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:30.488710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.502675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 294/593] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:30.508284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.606633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257935-8fa3-41af-8348-976b9cdea062 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.616333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54bae-b2f1-46da-9cdc-306bb9822745 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.634081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.645810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.646742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.652851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.652851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.654934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.655674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.656030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.656453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.656813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.657355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:30.657733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.658925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.659442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:30.672413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:30.673355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:30.674227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:30.677155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 300/594] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:30.679231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.686398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 295/595] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:30.687584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.698704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.795623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369f245-62b7-4e92-8aa4-1c9bf4d2072a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.798461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3baa8fa-e77f-4aa8-9555-fc9e775de761 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.829666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.830278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.830560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.830833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.831524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:30.831830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.833280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.833946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.834574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.834954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.835222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.845356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 296/596] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:30.846161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.851376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:30.852877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-544a1a0d-a968-425a-8406-1dcdc75b8a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:30.869321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:30.870289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:30.871165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:30.871165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:30.871165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:30.871165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 301/597] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:30.871822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc0d2-5b8e-48ed-ab46-f7986ab2fed8 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.871918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc0d2-5b8e-48ed-ab46-f7986ab2fed8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.872198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc0d2-5b8e-48ed-ab46-f7986ab2fed8 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.877421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 302/598] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:30.877421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc0d2-5b8e-48ed-ab46-f7986ab2fed8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:30.887711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.952044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:30.958448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb0f3-6e12-42dc-8071-816762359ac5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:30.994089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:30.994716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:30.994990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:30.995258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:30.996085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc0d2-5b8e-48ed-ab46-f7986ab2fed8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.006293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.007765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:31.023338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:31.024300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:31.025183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:31.025183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:31.025183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:31.025183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 297/599] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:31.025413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.042041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2050bb-d99a-48a4-8546-c011ddb4875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.043283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2050bb-d99a-48a4-8546-c011ddb4875f None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.043767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2050bb-d99a-48a4-8546-c011ddb4875f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.044446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2050bb-d99a-48a4-8546-c011ddb4875f None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/d83045f9a43b48d7bbcce339f7bfa5a2/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.067384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2050bb-d99a-48a4-8546-c011ddb4875f None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=d83045f9a43b48d7bbcce339f7bfa5a2, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.070498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2050bb-d99a-48a4-8546-c011ddb4875f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.116668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:31.131383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae4ce-64c4-4468-aef7-44834f0b8120 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.146772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 303/600] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:30 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/d83045f9a43b48d7bbcce339f7bfa5a2/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:31.157728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.158007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.158279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.165833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.166470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.166748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.167022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.167689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:31.167995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.170571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.187977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 298/601] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:31.187977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.209264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:31.210493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:31.292665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 304/602] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:31.293388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.314167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a918ad-86ad-47f1-a4c8-b8f6c681563c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.343803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:31.354424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.355346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.359947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.359947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.359947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:31.359947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.359947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.373137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 299/603] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:31.382014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.399435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:31.405998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2c37a-aa27-4fa5-b823-fa071a2cc003 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.444973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.445635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.446076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.446403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.455990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.458130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:31.472859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:31.473787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:31.474671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 305/604] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:31.480725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.506659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77149f1-8740-4e2b-b2d3-7fde15aa03c2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.603661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f1821-6ecb-4696-97eb-dd03f9cbb98f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.616643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:31.616900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.617511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.617796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.618077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.638244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.641448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.647942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.648674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.649061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.649508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.650222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:31.650619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.652185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:31.656217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:31.657248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:31.658273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 300/605] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:31.659101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.665457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 306/606] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:31.666282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.758302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:31.774229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6fff5-d200-4345-bd4d-f86760551b90 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.809670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2ebf-8949-4872-af21-03a535dfdad6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.810846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.811450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.811726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.811990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.812725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:31.813025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:31.814456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:31.827671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 307/607] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:31.828582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.856405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.857444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.857732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.858002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:31.863603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:31.871765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:31.878844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:31.879799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:31.880759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:31.884240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 301/608] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:31.886625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:31.955123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33bd7-5425-4e0b-ac23-27a9337b2e4f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:31.993358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:31.994354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:31.994815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:31.995296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/113ccc244a814d7580d1c9f6319edf04/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.011730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=113ccc244a814d7580d1c9f6319edf04, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:32.014240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:32.019448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000214-c3e4-41da-b868-f5b9264f6a67 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:32.046853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:32.055399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:32.056065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:32.056367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:32.056638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/b50c8a35e28e44bba3a4b4e56057a3fe/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.071648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=b50c8a35e28e44bba3a4b4e56057a3fe, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:32.073108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:32.118768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 308/609] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:31 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/113ccc244a814d7580d1c9f6319edf04/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:32.120120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:32.199930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 302/610] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:31 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/b50c8a35e28e44bba3a4b4e56057a3fe/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:32.262291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:32.291366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6281a30-cd47-4b80-8556-c4e995382855 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:32.340035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:32.340704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:32.340984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:32.341259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.342578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:32.342882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:32.344379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21d398-e37b-49f4-9d27-7ceb59341bf1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:32.354315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 309/611] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:32.360526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:32.482361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8366cb-be05-48f5-bc28-3bc3c5219404 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:32.525929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:32.526933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:32.527453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:32.527929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.543422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e636-7a77-41d6-9b39-5552c5c8b0a3 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:32.543755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e636-7a77-41d6-9b39-5552c5c8b0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:32.543988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e636-7a77-41d6-9b39-5552c5c8b0a3 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.547050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:32.547444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 310/612] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:32.549021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:32.551542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:32.551830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:32.552106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:32.569750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:32.570716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:32.571591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:32.571591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:32.571591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:32.574341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 303/613] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:32.584994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:32.589439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:32.590915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:32.633260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:32.701675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 311/614] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:32.760767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c887441-8efa-4b9d-8886-af4d95605aab None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:32.806791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:32.807399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:32.807676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:32.807948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:32.808623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:32.808925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:32.812532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:32.826955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 304/615] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:32.832570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:32.945347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:32.999544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e26fd0-733e-4b95-997f-1cf216d3262b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.048976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.049856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.050283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.050768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.053703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.059188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.061469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:33.073715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:33.074880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:33.076523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:33.078609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 312/616] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:33.084246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.231096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:33.244244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa8ab-1f86-46a2-b01c-412b88cba476 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.262080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c4625-0a1c-4360-96b5-de7651be4481 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.282162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.282807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.283089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.283368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.296016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:33.301194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:33.302158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:33.303045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:33.303838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 305/617] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:33.309189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.335950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:33.374009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.374623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.374900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.375166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.375899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:33.376220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.377695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.430471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 313/618] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:33.434281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0a7ac-8b1a-41f1-ba34-a7b3a57db51b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.441184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.471662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.472345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.472627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.472899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.473894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:33.474204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.475743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.485049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 306/619] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:33.495127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.520500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:33.580276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963ab9e-22d6-4c87-8ae2-00307477798e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.617323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.617973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.618253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.618523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.630289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.631925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:33.649073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:33.650100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:33.650994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:33.650994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:33.650994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:33.650994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 314/620] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:33.661045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.677038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb28ec3-2876-4116-a38f-c23dc18fef46 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.723375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.724033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:33.725608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.726207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.726738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.739550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.741385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1a4ecd3d-f224-4262-982d-a10efeb75890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:33.757222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:33.758226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:33.759111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:33.759111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:33.759111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:33.761300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 307/621] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:33.762789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b02183-57ca-4023-aefd-b502343a3487 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.763905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b02183-57ca-4023-aefd-b502343a3487 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.764459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b02183-57ca-4023-aefd-b502343a3487 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.766459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 308/622] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:33.768403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b02183-57ca-4023-aefd-b502343a3487 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.800340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03793bd-4d8c-4649-88a7-5c05bcbdc662 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.839274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.839806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.840088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.840409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.841263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:33.841581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.843434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e21e8d-af64-47e2-b0c0-9be892519f3e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.858013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:33.863999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 315/623] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:33.863999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.863999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.863999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.884309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:33.886174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:33.914113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b02183-57ca-4023-aefd-b502343a3487 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:33.959964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:33.961592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:33.962084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:33.962786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:33.968404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:33.968743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:33.970471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:33.981587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 316/624] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:33.983098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:33.986498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 309/625] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:33.995111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.040245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.156928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ebaca-74d2-4183-a8c9-e4d2bbbc2b96 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.182722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.196070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.196878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.197154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.201717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.201827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc833768-6936-4aa7-8fd6-9a124eb777c1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.206445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.207913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:34.221099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:34.222084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:34.223094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:34.227142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 310/626] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:34.228335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.262564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.263731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.270345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.270345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.271877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.273505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:34.286188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:34.287146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:34.288453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 317/627] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:34.289280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.342796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e49c1-5373-4718-bba3-22a13420c5bd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.392197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.392819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.393099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.393651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.394766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:34.395077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.396541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.398140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.413536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb50a30-ee98-47b6-9eb1-4a013f508ef1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.425041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 311/628] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:27:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:34.425041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.456407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.457419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.457970 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.458443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.469394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:34.472354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:34.474965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:34.475921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:34.477099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:34.479641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 318/629] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:34.480801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.550220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb782-c1d5-4c91-9175-ec8aa7a9630e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.597054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.598719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.599212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.599675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.600596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:34.600724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2cc2-26ca-42d0-a832-32136a02c029 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.601300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.603972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.619848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 312/630] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:34.621269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.645221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6a52-9e59-47b8-b86b-dd741228f824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.645964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6a52-9e59-47b8-b86b-dd741228f824 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.646310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6a52-9e59-47b8-b86b-dd741228f824 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.646652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6a52-9e59-47b8-b86b-dd741228f824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.675319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.740267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6a52-9e59-47b8-b86b-dd741228f824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.742314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6a52-9e59-47b8-b86b-dd741228f824 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.765352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.769310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae51ba-ab2d-4b68-a443-0c68123094d7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.806527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.807135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.807406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.807670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.808350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:34.808649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.810075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.818987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 313/631] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:34.821039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.861088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 319/632] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:34.861088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc10b7-23c6-4c2d-a1ff-b5fe37be99bf None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.861088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc10b7-23c6-4c2d-a1ff-b5fe37be99bf None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.861088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc10b7-23c6-4c2d-a1ff-b5fe37be99bf None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.861088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 320/633] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:34.861088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc10b7-23c6-4c2d-a1ff-b5fe37be99bf None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:34.928720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b273a7-d8cf-4c84-bc2b-0c1210776fbf None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.954013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:34.961892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250220c5-c90d-44a3-a554-125f2850beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.968355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc10b7-23c6-4c2d-a1ff-b5fe37be99bf None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:34.968485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250220c5-c90d-44a3-a554-125f2850beb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.968485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250220c5-c90d-44a3-a554-125f2850beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.968485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250220c5-c90d-44a3-a554-125f2850beb5 None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/256b9d949c8e4734a6bce965a107c060/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:34.977996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250220c5-c90d-44a3-a554-125f2850beb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=256b9d949c8e4734a6bce965a107c060, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:34.979400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250220c5-c90d-44a3-a554-125f2850beb5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:34.995082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:34.995704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:34.995983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:34.996335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.007902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.009423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:35.024062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:35.025271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:35.026409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:35.026409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:35.026409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:35.026409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 321/634] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:35.026409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.064085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 314/635] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:34 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/256b9d949c8e4734a6bce965a107c060/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:35.065431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd8977-3c05-4bdb-b94b-7a56f9988e67 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.065720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd8977-3c05-4bdb-b94b-7a56f9988e67 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.065991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd8977-3c05-4bdb-b94b-7a56f9988e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.085531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b0fd8977-3c05-4bdb-b94b-7a56f9988e67 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:35.086790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b0fd8977-3c05-4bdb-b94b-7a56f9988e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:35.095398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.149569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.152993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522426-92e0-4ccf-a023-4ccf2de96f8b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.166151 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 315/636] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:35.167642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e23f1-f037-4546-834b-a84a74d61118 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.167936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e23f1-f037-4546-834b-a84a74d61118 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.168242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e23f1-f037-4546-834b-a84a74d61118 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.172197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 316/637] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:35.172197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e23f1-f037-4546-834b-a84a74d61118 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.207499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.208574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.209045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.209516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.222089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.223913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-eb3449d4-0d49-4ebe-a002-ed7aba9820e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:35.241484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:35.242711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:35.243799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:35.243799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:35.243799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:35.248088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 322/638] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:35.249529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.249989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.250569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.271012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:35.272228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:35.301666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e23f1-f037-4546-834b-a84a74d61118 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.348045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.349005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.349547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.349823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.350476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:35.350784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.352280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.353818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 323/639] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:35.355035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.378192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 317/640] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:35.383079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.387588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.485483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.514694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cbf30-2864-477c-a1a0-0b628d951d78 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.524091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563361-51a8-42f5-b1da-cbf9fdd17b2d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.561163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.561774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.562056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.562330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.562533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.562978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:35.563274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.564013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.564519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.564706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.565169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.573822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.576233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.578934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 324/641] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:35.583415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:35.598112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:35.599129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:27:35.600690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:35.603418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 318/642] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:35.604557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.676177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.691486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a74a59-8d78-4708-a291-75841004c64d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.718561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005f02-f90e-4ef5-8179-1c0040131111 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.731192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.732497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.732783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.733055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.743660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:35.748935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:35.749934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 325/643] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:35.750779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.754544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.762858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.765524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.765524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.765524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.771573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.773049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:35.785097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:35.786001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 319/644] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:35.787039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.871994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b8eb-a607-45f4-8ba6-ff16e7488b0a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.893911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f17c72-f19b-412c-98d8-0f4ae572adc2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:35.907459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.908067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.908393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.908663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.909444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:35.909750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:35.911180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:35.918430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:35.933913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 326/645] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:35.933913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:35.936328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:35.937148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:35.937496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:35.937842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:35.949110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:35.954433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:35.955375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:35.956234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 320/646] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:35.956956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.014119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.058915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2642-d12c-4041-bef3-9659196be0a1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.066353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082425-5b14-488e-a49b-2e4a13629843 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.096020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.096649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.096927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.097197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.097846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:36.098148 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.099520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.100459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.101200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.101516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.101799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.102485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:36.102791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.104288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.108493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 327/647] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:36.109496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.118643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 321/648] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:36.119682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.167421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.218846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9569a7-2e53-43a3-ac55-360ba9e79be0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.225869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44734d3-26c1-4c55-82c9-3e57f3dcdb4c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.242804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.254908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.255667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.256065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.256403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.260932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.261583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.261886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.262195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.262908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:36.263731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.265434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.278535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 322/649] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:36.278535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.334250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.335717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.382903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418caf6-e6ae-46dc-8382-544eaa71c678 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.385280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.421151 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.422202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.422486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.422763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.433860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.435948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.453543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 328/650] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:36.453543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:36.453798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:36.454652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:36.455543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:36.455543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:36.455543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:36.455543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 323/651] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:36.455543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.480784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.562319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df19cb8-5948-4ccb-993e-40a6fc8f0e3f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.565408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29672b-391b-4275-b97b-eae4d1b4fa5a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.599934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.600343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.600833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.600833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.600978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.601185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.601282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.601480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.601807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=0cb8a6f42b6e482eb109be9d2f40e64f) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:36.602116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.603567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.616406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 329/652] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/role_assignments?user.id=0cb8a6f42b6e482eb109be9d2f40e64f&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:36.616406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.620466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.621841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:36.637833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:36.639477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:36.640756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:36.640756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:36.640756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:36.640756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 324/653] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:36.640756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.653664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.726063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.744331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457dbb6-398e-4acb-8926-e4c830959675 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.777887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68af8e6-8ff1-40eb-a1a2-56e96f2d7f43 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.784135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.784822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.785102 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.785373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.797529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.798751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:36.817695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:36.818861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:36.820029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:36.820029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:36.820029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:36.820029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 330/654] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:36.820029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.825752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.826608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.826888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.827161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.827811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:36.828115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.829516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:36.847705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 325/655] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:36.849768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:36.933129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:36.943472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07558abf-bbed-42eb-8fc5-f61346e94484 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.983096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4528f7-eca4-491e-8197-10070f325213 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:36.987610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:36.992582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:36.992582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:36.992582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:36.992582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:36.992582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:36.992582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.001772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.013907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 331/656] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:37.013907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.030270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.031167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.031599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.032029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.032852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:37.033311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.034911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.047474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 326/657] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:37.048661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.143079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08e1f3-834a-4ed5-a931-1d753c5190ff None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.158120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944ba31-99e0-4b1d-9067-ac416a5f4860 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.186279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.192115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.192854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.193188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.196891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.197026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.197026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.197026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.197026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.202839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.204555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.205753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.207374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:37.218817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:37.220442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 14:27:37.221585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:37.223007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:37.223881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 14:27:37.224846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:37.226483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 332/658] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:37.231560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.232015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 327/659] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:37.233209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.311039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.346577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0d296-c49a-4e84-94f2-6f036b77d20a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.369415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72830be-516b-4dcd-9cd9-7926ca30c262 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.382811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.383671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.384133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.384656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.385580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:37.386065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.387847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.407386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 328/660] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:37.408121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.415070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.415723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.416000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.416303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.424431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.425880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5ec46575-c661-4553-a1ef-dc2f6663c8a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:37.439019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:37.440084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 333/661] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:37.441463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12757cd-a982-47b2-8bd5-8a80bc482646 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.442009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12757cd-a982-47b2-8bd5-8a80bc482646 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.442009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12757cd-a982-47b2-8bd5-8a80bc482646 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.447264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 334/662] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:37.447264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12757cd-a982-47b2-8bd5-8a80bc482646 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.466904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.521581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2d64b-137c-4519-ab3f-5887852caca5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.555079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12757cd-a982-47b2-8bd5-8a80bc482646 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.556624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.557140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.557421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.558324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.558434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:37.558739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.560840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.575353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.578364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 329/663] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:37.579297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.596286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.597161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.597447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.597720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.674360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.675784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.686421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e56660-d20a-474e-89ce-f899f3008554 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.688956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.721007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.721611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.721887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.722162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.722841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:37.723145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:37.724592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c7fb-4372-42bc-b7b6-ffdbc3fd0658 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:37.737805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 330/664] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:37.739237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.739521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.739790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.758842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:37.760044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:37.790944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 335/665] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:37.792289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.810178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.829003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 331/666] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:37.829856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:37.897270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eaa73-8850-4514-a4df-0749a847fe8c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:37.940958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:37.941590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:37.942334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:37.942677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:37.951792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:37.959752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b5016-c700-4696-a19b-dcca6ca7cfa8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.006443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.007321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.007600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.007876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.008862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=d83045f9a43b48d7bbcce339f7bfa5a2) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:38.009171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.011362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.019951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 332/667] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/role_assignments?user.id=d83045f9a43b48d7bbcce339f7bfa5a2&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:38.021516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.037318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.038849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.064637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.165517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 336/668] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:38.167350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0a709-5533-41c9-9bc1-e89a6b365b99 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.176183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.209782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.210342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.210667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.210941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.226745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.228982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:38.251895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:38.252928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:38.253799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:38.253799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:38.253799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:38.254413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 333/669] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:38.254943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.270289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.322923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50345-365b-480f-b302-ef614d8619bd None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.346171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.368921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.369561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.371215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.371215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.384435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.385905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.391411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a0bb4-6dee-41fb-8691-f370af32eb71 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:38.401710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:38.402707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:38.404202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:38.404202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:38.404202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:38.404202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 337/670] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:38.404894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.438892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.440634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.441072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.441515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.455533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:38.462887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:38.463880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:38.464771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:38.468115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 334/671] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:38.469516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.513924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70202a-7be0-4fb9-96d7-5c1076d8f2b8 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.552170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.553118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.553566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.553888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.554603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:38.554988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.556763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.562272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.574615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 338/672] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:38.576044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.603534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ef809-60fd-4838-970b-636ce22e623e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.617921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.641774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.642841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.643363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.643846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.644737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:38.645240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.646798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.662030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 335/673] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:38.663289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.704983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e17d51-f2b7-41e4-9050-99be80c4f54d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.750208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.751042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.751328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.751596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.752283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] RBAC: Adding query filter params (name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:38.752584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.754231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.768249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 339/674] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:38.768249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.779477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59ffe-df2a-442e-8e2c-4783d5dfd162 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.810126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.818414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.819210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.819590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.819961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.828256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.830021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:38.842390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:38.843457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:38.844599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:38.846360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 336/675] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:38.847360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:38.893543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:38.908661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64787e-bbbe-4592-a036-79c9ab2a3264 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.957175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:38.958058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:38.958753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:38.959177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:38.962381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44a292-a293-4ccd-815a-26d7e1d479e0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:38.969915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:38.971555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:38.987544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:38.988646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:38.989731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:38.991968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 340/676] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:38.993194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.008919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.009643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.010024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.010492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.023515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.025037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:39.039259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:39.040254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:39.041168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:39.041168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:39.041168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:39.045793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 337/677] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:39.046340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.094142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:39.108294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2182-4b79-4c64-98e5-73d4a421c48a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.147532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.148166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.148463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.148729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.149422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:39.149732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.151173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f14a-ea42-40fb-9c1f-c46b4cc8f302 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.158938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c0c45-7a9c-473d-b279-cdb8ed1dad9e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.165122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 341/678] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:39.166299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936306f-c1b2-4119-a34d-1c8a366f0b95 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.166581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936306f-c1b2-4119-a34d-1c8a366f0b95 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.166854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936306f-c1b2-4119-a34d-1c8a366f0b95 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.168318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 342/679] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:39.173373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936306f-c1b2-4119-a34d-1c8a366f0b95 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.189190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:39.199558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.200197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.200507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.200781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.201468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:39.201773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.203193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.216038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 338/680] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:39.216931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.287580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936306f-c1b2-4119-a34d-1c8a366f0b95 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.324507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b2353-0e2e-4fa6-8151-6f25c58a43e7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.324703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.325151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.325507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.326007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:39.326110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.327083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:39.327385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.328777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f66293-49ae-4af7-8ef3-aeda32cca96d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.345094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 343/681] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:39.345094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.345094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.345094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.360726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:39.361908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:39.364664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.365296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.365568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.365820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.366532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=b50c8a35e28e44bba3a4b4e56057a3fe) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:39.366835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.368377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.384176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 339/682] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/role_assignments?user.id=b50c8a35e28e44bba3a4b4e56057a3fe&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:39.384176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.459732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 344/683] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:39.460893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.463518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:39.573624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29e26-3b47-4e8c-982e-936440146923 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.593444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1c170-a5f7-456d-b163-208e257036ae None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.617279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.618222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.618882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.619399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.620499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=113ccc244a814d7580d1c9f6319edf04) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:39.621011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.622689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.629175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 340/684] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/role_assignments?user.id=113ccc244a814d7580d1c9f6319edf04&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:39.632209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.632827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.633099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.633364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.642175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.643765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:39.657500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:39.658529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:39.659465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 345/685] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:39.666412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.707236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:39.784706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.796370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2899b-27c7-4bb7-9d6f-a8ece1ee842e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.865630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.866483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.867001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.867476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:39.869271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:39.869821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:39.871902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:39.887036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:39.897377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 341/686] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:39.903464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:39.936217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14016d3-098b-46d6-907c-e529756ea1c2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:39.991329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:39.992610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:39.993112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:39.994422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.008729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:40.015142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:40.018140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:40.019085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:40.019940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:40.026912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 346/687] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:40.027716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.038340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c9dd-9bb1-4efe-a75c-77365ea34bd0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.080867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.081480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.081754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.082023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.172212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.174009 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.174441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af618846-4485-4a93-ac59-91ee53c48cda None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.201313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.216669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.217156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.217439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.217709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.218376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:40.218738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.220295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.233187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 347/688] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:40.239069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.272399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.287806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 342/689] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:27:40.297283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.349880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0960eda-d1c0-4f65-808d-9ed8031342f2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.392531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.393572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.394052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.394523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.406272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.408233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:40.424260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:40.425212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:40.426095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:40.426095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:40.426095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:40.432168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 348/690] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:40.433799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.472269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.482468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7f75e-43d9-4afc-875e-452cd0a84e5f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.519018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.519677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.519958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.520257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.531326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.532726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.545135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df46ed-91bc-48f3-9a7d-ceb6b6d7f035 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-89279829-fdc6-42f7-8a4c-411b8723a531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:40.548434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:40.549397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:40.550379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:40.550379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:40.550379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:40.550379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 343/691] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:40.550986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ceaf-0d8b-46b3-9110-1dce16f24199 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.551230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ceaf-0d8b-46b3-9110-1dce16f24199 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.551504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ceaf-0d8b-46b3-9110-1dce16f24199 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.556958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 344/692] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:40.557742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ceaf-0d8b-46b3-9110-1dce16f24199 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.588297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.589365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.589888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.590364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.592023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:40.592591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.594446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82d8a-13dd-4b24-8aa1-c323f84e597d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.605123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.609836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 349/693] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:40.615577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.621823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.621823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.635329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:40.636819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:40.668185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ceaf-0d8b-46b3-9110-1dce16f24199 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.704723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.705432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.705795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.706208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.706951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:40.707396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.708911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.713386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 350/694] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:40.714401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.733875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 345/695] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:40.735019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.743268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.853168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0866175-da37-4bbb-be30-bca04dea1b14 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.864587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:40.868678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ce5de-7471-4728-bb0d-32d072ae7aa3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:40.921450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.922094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.922375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.922647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.931196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.932918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.935777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:40.937344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:40.937867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:40.938334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:40.953331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:40.955272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:40.960196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:40.961138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:40.962075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:40.965127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 351/696] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:40.971524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:40.978458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:40.979373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:40.980273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:40.990295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 346/697] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:40.995659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.141245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:41.143436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e4daf-b055-4312-9a8a-fc6b79e8471d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.163678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f8e64-dfa9-4b8b-ad40-7b484e7d2ea2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.182381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.183282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.183787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.184298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.185349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:41.186021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.187652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.200674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 347/698] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:41.201842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.205426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.205980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.206262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.206537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.208202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:41.208520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.209951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.227699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:41.234700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 352/699] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:27:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:41.235921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.317891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b968a-46b1-40f1-a392-f9f0147af5c3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.356450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.357470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.357986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.358555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.359438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.361061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.365523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5249fd-efa9-4926-85d9-51fd229c3b97 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:41.373849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:41.374845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:41.375673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:41.378587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 348/700] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:41.379594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.413331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.414185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.414463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.414743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.415885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=256b9d949c8e4734a6bce965a107c060) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:41.416225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.417967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.426374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 353/701] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/role_assignments?user.id=256b9d949c8e4734a6bce965a107c060&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:41.427272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.441914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:41.485285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:41.507422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d654-99ca-462d-905b-b81657856e9b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.553466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.554095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.554366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.554633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.555045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b3739-130b-4fa7-8905-72a45cc3efcd None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.594511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.595671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.596173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.596715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.597564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:41.598150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.599945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.605794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 354/702] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:41.612374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.640518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.642159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.694944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:41.738316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6992c-a87e-4b68-81da-84ad0a7c3ac9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.755796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:41.781852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 349/703] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 14:27:41.787796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:41.796414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.797768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.798252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.798727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.800081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:41.800597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.802856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.811580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 355/704] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:41.906224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728fda1-11a5-4305-94cb-42377160576a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:41.945572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:41.946227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:41.946504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:41.946774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:41.960042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:41.961561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cf783767-bf6d-4e2d-a5ad-9e98a479a9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:41.979402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:41.980424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:41.981327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:41.981327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:41.981327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:41.981327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 350/705] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:41.987272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:42.112399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265c4f-1b6a-4853-8baf-71c6353cbbf6 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:42.125819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54c131-4590-41d7-90bc-618cc33a5d5b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.126103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54c131-4590-41d7-90bc-618cc33a5d5b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.126370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54c131-4590-41d7-90bc-618cc33a5d5b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.127710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 351/706] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:42.133555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.133837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.134105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.159199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:42.165229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.165351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:42.165453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.165453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.165453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:42.165689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:42.165784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:42.165784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:42.167697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:42.181466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 356/707] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:42.189139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:42.202478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:42.252288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 352/708] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:42.328477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97ff3f-7ae1-434a-b869-24dc7d487f31 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:42.382748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:42.383356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.383570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.383843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.395551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:42.397054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-9a3c412a-ddce-4cb7-833c-1b55b1ca4122 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:42.412805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:42.413749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:42.415147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 357/709] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:42.419815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:42.543206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903bd35-e835-4a93-9510-1db336165f3d None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.543655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903bd35-e835-4a93-9510-1db336165f3d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.544066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903bd35-e835-4a93-9510-1db336165f3d None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.545804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 358/710] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:42.550990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.551571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.551733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.561174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305e33d-0a82-47c4-99e9-684e7502ff39 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:42.575274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:42.577392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:42.600575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:42.601336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.601741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.602032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.602735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:42.603042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:42.605047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:42.620126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 353/711] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:42.625557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:42.642191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:42.655148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:42.657312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 359/712] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 14:27:42.658347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:42.787618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f51c6e-3e37-4573-b1db-bb1e2949fc43 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:42.821219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1768e5-a343-44dd-8c78-f30b2fd62c38 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:42.857394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905620d4-43c6-4282-9999-0708701a391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:42.858007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905620d4-43c6-4282-9999-0708701a391d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.858287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905620d4-43c6-4282-9999-0708701a391d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.858558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905620d4-43c6-4282-9999-0708701a391d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.859279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905620d4-43c6-4282-9999-0708701a391d None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=0cb8a6f42b6e482eb109be9d2f40e64f) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:42.859587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905620d4-43c6-4282-9999-0708701a391d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:42.861059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905620d4-43c6-4282-9999-0708701a391d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:42.866907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 360/713] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:42 2022] GET /identity/v3/role_assignments?user.id=0cb8a6f42b6e482eb109be9d2f40e64f&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:42.867466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:42.868112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:42.868410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:42.868678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] PATH_INFO: `/v3/services/network` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:42.869261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:42.870603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:42.880331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:42.881491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:42.882348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:42.885269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 354/714] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:42 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:42.896280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905620d4-43c6-4282-9999-0708701a391d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:42.959410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:43.020833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:43.105996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:43.114999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905620d4-43c6-4282-9999-0708701a391d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:43.123096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c811d-b039-4c40-91e6-3bf28d380247 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:43.184309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:43.187417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:43.188210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:43.191433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:43.191881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:43.194089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] RBAC: Adding query filter params (name=network) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:43.194624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:43.195042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:43.195526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:43.195933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:43.197503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:43.210822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 361/715] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:27:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:43.212972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:43.214562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:43.216338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:43.239556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:43.240799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:43.241918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:43.241918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:43.241918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:43.243743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 355/716] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:43.258112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:43.414351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37448491-bb46-4bd2-a575-544a38ca705e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:43.430667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5a142-c041-45dd-b02f-e2d140577bf7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:43.447035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:43.487943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:43.488583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:43.488857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:43.489123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:43.489755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] RBAC: Adding query filter params (type=network) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:43.490051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:43.491414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:43.498667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 362/717] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:27:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:43.505010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:43.505658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:43.505935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:43.506202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:43.506876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:43.507175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:43.508730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7b66a-21d1-4fcc-8e59-4b5ab9bc0d41 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:43.530715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 356/718] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:43.536281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:43.668272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e778c-32dd-4d21-aa44-50b2c9c2a625 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:43.719138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89044957-56b8-49c2-b990-d22085c0c8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:43.719672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89044957-56b8-49c2-b990-d22085c0c8d9 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:43.719946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89044957-56b8-49c2-b990-d22085c0c8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:43.720238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89044957-56b8-49c2-b990-d22085c0c8d9 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:43.720888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89044957-56b8-49c2-b990-d22085c0c8d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:43.722405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89044957-56b8-49c2-b990-d22085c0c8d9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:43.838892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 363/719] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:27:43 2022] POST /identity/v3/users => generated 254 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:27:44.442318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c17e56-9a44-4643-a620-50e564af98e3 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:44.442882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c17e56-9a44-4643-a620-50e564af98e3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:44.442882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c17e56-9a44-4643-a620-50e564af98e3 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:44.444525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 357/720] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:44.451567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7419f-e3b0-4948-b228-2133e5e70c61 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:44.452062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7419f-e3b0-4948-b228-2133e5e70c61 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:44.452186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7419f-e3b0-4948-b228-2133e5e70c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:44.471692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-80d7419f-e3b0-4948-b228-2133e5e70c61 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:44.472928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-80d7419f-e3b0-4948-b228-2133e5e70c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:44.500427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:44.544091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 364/721] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:44.553390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09687cec-e06b-4c9a-b7dd-063435844ad8 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:44.553677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09687cec-e06b-4c9a-b7dd-063435844ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:44.553952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09687cec-e06b-4c9a-b7dd-063435844ad8 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:44.555272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 358/722] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:44.560774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:44.561057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:44.561328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:44.579920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:44.582090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:44.677805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 365/723] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:44.851911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09687cec-e06b-4c9a-b7dd-063435844ad8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:44.854925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:44.978300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09687cec-e06b-4c9a-b7dd-063435844ad8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:45.030148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:45.031147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.031627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.032218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.043261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:45.049188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:45.050171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:45.051035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:45.053760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 359/724] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:45.059857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.070187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.182484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:45.190498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a3364-55de-4102-8a5b-272ee136d1ac None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:45.203961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:45.247855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:45.248490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.248763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.249028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.249692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:45.249990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:45.251368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:45.266086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 366/725] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:45.268484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dc5f9-4000-4f29-9fb5-5fc37597c528 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:45.276991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.316442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6da0a-14db-45cb-a3fd-09d1d0fb6412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:45.317055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6da0a-14db-45cb-a3fd-09d1d0fb6412 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.317393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6da0a-14db-45cb-a3fd-09d1d0fb6412 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.317666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6da0a-14db-45cb-a3fd-09d1d0fb6412 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.318829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6da0a-14db-45cb-a3fd-09d1d0fb6412 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:45.320262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6da0a-14db-45cb-a3fd-09d1d0fb6412 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:45.437024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b976cb-52a6-4d13-92f6-b684ccdde5f3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:45.438490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 360/726] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:27:45 2022] POST /identity/v3/services => generated 251 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:27:45.454949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdeb46-a4c6-4e64-bf36-c1e71299bec1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.455598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdeb46-a4c6-4e64-bf36-c1e71299bec1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.455598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdeb46-a4c6-4e64-bf36-c1e71299bec1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.458176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 361/727] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:45.462712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.463001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.463263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.490831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:45.492062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:45.504644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:45.505394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.505673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.505936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.529513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:45.530993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:45.551372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:45.552447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:45.553358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:45.553358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:45.553358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:45.553358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 367/728] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:45.558764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.566385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:45.571743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 362/729] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:45.711255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e2fe-49f3-4f68-bcc4-ef2d75e26308 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:45.751480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:45.752092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.752397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.752660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.753356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:45.753657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:45.755021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:45.768821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 368/730] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:45.774239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.830833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:45.865904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.878942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d47ae8-068b-430b-bc94-659011d73b9c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:45.915134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:45.915770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:45.916040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:45.916464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:45.925079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:45.926089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:45.938239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:45.939193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:45.940104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 363/731] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:45.947474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:45.956193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:46.033308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d9e68-4a50-4b80-8e85-1b6ebf306c3f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.054539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3b922-1f03-4f43-9e47-a3b8883405a9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.068701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:46.090673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.092469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.092741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.093003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.093724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:46.094021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:46.095427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:46.108114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.108775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.109049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.109316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.128830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 364/732] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Sep 7 14:27:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:46.132373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:46.135558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:46.147838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:46.148841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:46.149698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:46.151361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 369/733] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:46.157170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:46.264361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cfd9d-1b87-4636-9122-d06f4535d9cc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.301610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548bfef-f149-474b-bf91-e8afb64808c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.302229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548bfef-f149-474b-bf91-e8afb64808c7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.302504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548bfef-f149-474b-bf91-e8afb64808c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.302772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548bfef-f149-474b-bf91-e8afb64808c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.345598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:46.368324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd005e29-e372-4bec-a721-159ec5424710 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.376044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548bfef-f149-474b-bf91-e8afb64808c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:46.378533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548bfef-f149-474b-bf91-e8afb64808c7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:46.401087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:46.415221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.416017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.416486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.416914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.417932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:46.419148 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:46.420825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda719d-e563-4374-9191-0373bbe48e1d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:46.432186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 370/734] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:46.433724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10047-4f95-475d-a9cd-21ae31d55ca2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.434140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10047-4f95-475d-a9cd-21ae31d55ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.435055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10047-4f95-475d-a9cd-21ae31d55ca2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.437481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 371/735] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:46.439000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10047-4f95-475d-a9cd-21ae31d55ca2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:46.508335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 365/736] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 14:27:46.510279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.510691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.511090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.537922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:46.540254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:46.573279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10047-4f95-475d-a9cd-21ae31d55ca2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.615827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.616838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.617546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.618331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.620793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:46.626446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 366/737] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:46.628972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:46.640924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:46.648291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:46.677602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:46.678755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:46.679719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:46.679719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:46.679719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:46.688656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 372/738] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:46.694750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:46.788175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:46.831933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663a545-496b-4e89-8387-bd40b8799e5f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.871197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.872639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.873059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.873496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.873812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4747e-cf52-4c79-a96e-a15c295c5c93 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:46.884209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:46.886046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-487d2e5e-917b-447e-85b3-74452b6cc3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:46.904373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:46.905329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:46.906247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:46.906247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:46.906247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:46.908996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 367/739] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:46.910374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257d7fd-7e1e-4ede-926e-771e58223af2 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.910777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257d7fd-7e1e-4ede-926e-771e58223af2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.911190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257d7fd-7e1e-4ede-926e-771e58223af2 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.912824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 368/740] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:46.914418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257d7fd-7e1e-4ede-926e-771e58223af2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:46.917010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:46.917842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.918222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.918596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.920464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:46.920464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:46.922013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293799e-c323-4fad-b5f8-b3a8633be4a8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:46.935277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 373/741] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:46.941321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:46.941537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:46.941943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:46.959846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:46.961074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:46.976059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.030673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257d7fd-7e1e-4ede-926e-771e58223af2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.044259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 374/742] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 14:27:47.046057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.067325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.067930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.068250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.068522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.069165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:47.069465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.070817 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.085332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 369/743] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:47.086155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.091475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.178531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b7ac-1ecb-43ee-9329-cfff0b652766 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.207789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63c446-ca11-4623-8367-f6a0b2fc9429 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.215039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.215697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.215976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.216285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.223772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.225252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.225515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:47.239500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:47.240506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:47.241439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:47.244929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 375/744] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:47.246269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.256582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.257911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.258345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.258794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.260199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.262229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.344309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.356888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 370/745] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:27:47 2022] POST /identity/v3/services => generated 240 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:27:47.357381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536d030-dcfa-4f22-a207-6fcd3d06937a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.358945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.413864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.414508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.414782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.415048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.423818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.425393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:47.439266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:47.440306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:47.441237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:47.445656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 376/746] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:47.446586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.503425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7186062-0e54-4580-bf7d-694efdd47453 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.542621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.544629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.545112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.545604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.546658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:47.546965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.548440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.563598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2427b-2c4d-4a97-93af-7f0cf3c304fe None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.567774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 371/747] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:47.568595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.608060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.611178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.611912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.612311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.612656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.625517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:47.630505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:47.631441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:47.632587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 377/748] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:47.633707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.640242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.693792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0852-0dc5-468a-a1c0-07c1af2c1788 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.734486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.735108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.735391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.735917 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.737182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:47.737501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.739488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.755712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 372/749] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:47.758212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31194951-dd88-48cf-8c75-fc6fca600185 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.761460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.799186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.799955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.800332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.800674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.811844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.813432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:47.829034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:47.830026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:47.831216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:47.831216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:47.831216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:47.834906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 378/750] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:47.836012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.860625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.873117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828c2cf-e367-43e5-9112-6ee0051c2094 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.898324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:47.913939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.914976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.915554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.916364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.917285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:47.917767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.919377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.932967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 373/751] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:47.934118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:47.945549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924c8a6-329a-439c-bdd3-42431d98c59a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:47.984079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:47.984862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:47.985265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:47.986323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:47.987057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=256b9d949c8e4734a6bce965a107c060) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:47.987364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:47.988836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:47.995707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 379/752] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/role_assignments?user.id=256b9d949c8e4734a6bce965a107c060&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:47.997424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.064028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236ea6-5d71-4ed3-9e56-7aa2e7803ebf None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.103367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.104411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.104968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.105527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.106448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:48.107048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.108693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561eb1-aa3b-4049-9497-c372c8073b16 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.127758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 374/753] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:48.129832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa75af1-e639-4c3f-9ace-cd024e898d35 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.130396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa75af1-e639-4c3f-9ace-cd024e898d35 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.130883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa75af1-e639-4c3f-9ace-cd024e898d35 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.132778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8a7fe-6507-4b72-85bb-90e64d994b96 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.133236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 375/754] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:48.134364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa75af1-e639-4c3f-9ace-cd024e898d35 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.164566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:48.165315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:48.183026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.183660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.183939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.184238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.202617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.204343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-23486451-27e8-4e83-a688-e02eb9412c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:48.221197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:48.222205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:48.223082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:48.223082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:48.223082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:48.223082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 380/755] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:48.224487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.228008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.228008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.246894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:48.248757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:48.257630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa75af1-e639-4c3f-9ace-cd024e898d35 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.295342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.295982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.296357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.296629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.303598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.305063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:48.316543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:48.318587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 376/756] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:48.320873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.327029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 381/757] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:48.328895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.444533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e3a2e-5d9a-4505-8832-06f16ee8305a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.445956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:48.483245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.483855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.484130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.484424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.486116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:48.486439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.487948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.501244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 382/758] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:48.505259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:48.507113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.563873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a8d8-9ab8-4b42-b6b1-e8cf6670b90b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.629783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.630409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.630692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.630966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.631616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:48.631921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.633171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987aa94-5bca-46a8-a9d5-bf237c4fe4ea None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.633605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.647038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 377/759] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:48.660534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.676191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.676888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.677169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.677444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/0cb8a6f42b6e482eb109be9d2f40e64f/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.679377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=0cb8a6f42b6e482eb109be9d2f40e64f, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.680857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.762199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 383/760] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:48 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/0cb8a6f42b6e482eb109be9d2f40e64f/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:48.763674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.808259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91ec0c-1f83-49d9-8ed1-0c9edeab24e1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.828728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:48.873173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:48.883421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.884170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.884468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.884738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:48.885325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:48.886705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:48.901527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:48.902544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:48.904008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:48.912942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 378/761] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Sep 7 14:27:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:48.926506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:48.953273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39075f-8dbc-4016-8941-1d659671247a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:48.993154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:48.994358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:48.994647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:48.994923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.003023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.004886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:49.020237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:49.021448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:49.022676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 384/762] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:27:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:49.029927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.112689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5a7c-10da-4e63-921e-f81b01877f73 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:49.132609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:49.152578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147905f-3e18-4e84-90b0-cb41eb39e5c5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:49.190734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:49.191445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:49.191910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:49.192275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.193207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:49.193593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.195107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.197178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:49.197795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:49.198067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:49.198337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.199139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:49.199440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.207316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.214190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 385/763] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:49.221002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.221773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 379/764] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Sep 7 14:27:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:49.237813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.346922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5dc0-c185-4328-8a7b-2170a860d0f9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:49.387028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:49.387723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:49.388002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:49.388487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.393161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:49.400974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:49.448827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773a0c-bcd2-450f-9087-6241eb1d5fa1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:49.472929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.474576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.541415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:49.542041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:49.542323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:49.542595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.543225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:49.543526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.545471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.562482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 380/765] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:49.569652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.593487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 386/766] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:27:49.601341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.664288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:49.729229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c928d94-70c3-4feb-bd54-64da7acf0a36 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:49.767959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:49.772269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:49.772559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:49.772844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.792776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23a082-fc22-4635-b432-4d994405d089 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:49.795448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.796976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:49.830581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:49.833000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:49.835205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:49.835205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:49.835205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:49.835205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 387/767] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:49.838546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.848558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:49.870137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:49.870587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:49.870870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:49.871143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:49.871890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] RBAC: Adding query filter params (interface=public, service_id=9edb6d8db61a4a8aae765f9d406c6bed, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:49.875763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:49.877178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:49.899194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 381/768] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/endpoints?service_id=9edb6d8db61a4a8aae765f9d406c6bed&interface=public®ion_id=RegionOne => generated 194 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:49.920803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:49.997774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:50.007043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413ecc3-fa05-4e89-8ce6-633fcce15ac7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:50.051485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33d0a6-c8c3-4c89-93e3-23cd7b02e9d4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:50.055788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:50.057084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.057781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.058468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.060678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:50.061194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:50.063637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e8184-75ec-46b1-b317-3c240f54d2e5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:50.081599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 388/769] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:50.083528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370c41-d959-4952-9fd7-df73613a2b22 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.084413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370c41-d959-4952-9fd7-df73613a2b22 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.084996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370c41-d959-4952-9fd7-df73613a2b22 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.087685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 389/770] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:50.088993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370c41-d959-4952-9fd7-df73613a2b22 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:50.089340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf9bd7-9f83-47b2-9f18-1599147e94c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:50.090054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf9bd7-9f83-47b2-9f18-1599147e94c8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.090730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf9bd7-9f83-47b2-9f18-1599147e94c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.091104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf9bd7-9f83-47b2-9f18-1599147e94c8 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.092014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf9bd7-9f83-47b2-9f18-1599147e94c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:50.093662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf9bd7-9f83-47b2-9f18-1599147e94c8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:50.099461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 382/771] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:27:49 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:50.105773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252313f-3a29-45a5-b545-257bada70065 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.106204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252313f-3a29-45a5-b545-257bada70065 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.106577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252313f-3a29-45a5-b545-257bada70065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.144822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8252313f-3a29-45a5-b545-257bada70065 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:50.146224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8252313f-3a29-45a5-b545-257bada70065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:50.217873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370c41-d959-4952-9fd7-df73613a2b22 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:50.261892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:50.263411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:50.264449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.265295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.265717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.274634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:50.276644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:50.292133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:50.293337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:50.294967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:50.297969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 390/772] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:50.303762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:50.324184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 383/773] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:50.356903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:50.422815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10b6f-f368-4007-849a-5ad6fc50d88e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.423128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10b6f-f368-4007-849a-5ad6fc50d88e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.423477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10b6f-f368-4007-849a-5ad6fc50d88e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.425184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 384/774] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:27:50.430096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.430453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.430796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.454386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:50.455845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:50.465726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9f4a-369e-40a6-953f-0d781034880c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:50.508269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:50.509100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.509352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.509623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.510313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:50.510616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:50.512659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:50.528983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 391/775] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:50.534647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:50.538355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 385/776] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:27:50.642448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:50.652130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:50.675712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c12747-8103-4991-9b87-d67b3d7a6b99 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:50.732905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:50.733528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.733804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.734070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.734796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=438dc1eb842e4106b704a0a3c2a88fc6) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:50.735100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:50.736610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:50.743921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 392/777] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:50 2022] GET /identity/v3/role_assignments?user.id=438dc1eb842e4106b704a0a3c2a88fc6&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:50.855918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eba5a-d917-4b8a-a840-db364f615738 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:50.871327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:50.901910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:50.902573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:50.902852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:50.903649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:50.904288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:50.905700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:50.917922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:50.919558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:50.924906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:50.927640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 386/778] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:50 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:50.933816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.004758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.101782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafa1c7-d183-4e26-a186-48c7e83cfef9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.112010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44d916-c768-45fa-9bb1-9221964d8aa9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.150028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.150675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.150951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.151224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.174652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:51.180419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:51.181473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:51.182303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:51.183113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.183779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.184199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.188474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.189272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:51.189590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.191025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.193387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:51.194036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 387/779] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:51.196672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.199385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 393/780] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:27:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:51.205344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.272515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:51.352959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917e11-2991-453f-bf2e-463ba85030a3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.367038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9891791-abac-4738-b842-f38ae477e3c5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.391059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.391844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.392251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.392525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.393190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:51.393491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.394855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deccd38-b2c1-4b6f-9047-eb33bc73d2f4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.408487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 388/781] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:51.409916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfb9ba-fdeb-4d31-804f-a9b3f33b1ccc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.410204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfb9ba-fdeb-4d31-804f-a9b3f33b1ccc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.410481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfb9ba-fdeb-4d31-804f-a9b3f33b1ccc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.412765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 389/782] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:51.416858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.419924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfb9ba-fdeb-4d31-804f-a9b3f33b1ccc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.420359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.420832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.421255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.422063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] RBAC: Adding query filter params (type=network) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:51.422515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.424330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faa25c-89e2-4120-87f6-ffdc85643754 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.430024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 394/783] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:51.431600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.431829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.432101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.451542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:51.452752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:51.517527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:51.532447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 395/784] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:51.533527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.551115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:51.564471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfb9ba-fdeb-4d31-804f-a9b3f33b1ccc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.629916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.630794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.631594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.632766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.655454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.658945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:51.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:51.684060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:51.685537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:51.685537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:51.685537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:51.685537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 390/785] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:51.690528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.712492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44171-1cb7-4699-965b-dbf40479f981 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.751533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.752763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.753245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.753731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.754640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] RBAC: Adding query filter params (interface=public, service_id=229aa95e367b4670b2e0f3c8bc73fcd4, region_id=RegionOne) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:51.755147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.756816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.766825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 396/786] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/endpoints?service_id=229aa95e367b4670b2e0f3c8bc73fcd4&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:51.773890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.883341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:51.891030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f4643-fde3-4a1b-8a19-e08cca80e27e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.926496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb981f-4b23-4c00-9804-ffb6aa15d734 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:51.927766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.928943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.929426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.929905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.930743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:51.931243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.932899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.946038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 391/787] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:51.947072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:51.960034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:51.962722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:51.963324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:51.963600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:51.963869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:51.967891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:51.969852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:51.980674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 397/788] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:51.982191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.097483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873cd4-dc9e-4e2b-bd47-b19291c74195 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.107810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5a0e-a89e-4a50-b2ec-559a42e7e6dd None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.133905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:52.147204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.148095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.148550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.148972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.161464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.162422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.162853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.163274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.164839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.168033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.188048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:52.194260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:52.195424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:52.196524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:52.198916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:52.200002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:52.201131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:52.203502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 398/789] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:52.205183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 392/790] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:52.209158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.211209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.302953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:52.333765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf255ce-8049-42e9-9d36-5b326d74de9d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.334149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64344c72-c162-48d8-b90e-0cfee049cab8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.371275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.372080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.372411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.372527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.372698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.373023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.373303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.373408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:52.373572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.373687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.374642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:52.374941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.375116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.376329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.385196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 393/791] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:52.391172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.395074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 399/792] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:52.400614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.446099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:52.507401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e74162-d5ec-47ce-a2d9-d7ab6eb93344 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.549896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.550532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.550814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.551086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.568575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.570026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:52.598269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:52.599285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:52.600200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:52.600200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:52.600200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:52.605687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 400/793] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:52.609364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13ced-f9e6-48ff-a6da-6396f6b9f8c0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.611585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.669074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:52.677801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.678596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.678779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.678996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.702669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.704179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:52.729309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:52.730309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:52.731757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:52.731757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:52.731757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:52.732376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:52.736721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 394/794] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:52.742407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d43a-f388-4cdf-8ac2-03f09547a275 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.747732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.780498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.781034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.781315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.781589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.782237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:52.782542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.783934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.796838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 401/795] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:52.802933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:52.911721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1b3b2-d37a-49eb-84c7-b400437c8919 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.951221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:52.951889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.952191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.952469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.953165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:52.953405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:52.954781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a330-d01e-495b-a16c-1353257b04dc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:52.956600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65f254-db66-41fa-986a-fb9e5a5a22c1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:52.967447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 395/796] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:52.968751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf9a91-a44c-4b2d-b900-6885669f04f5 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.969038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf9a91-a44c-4b2d-b900-6885669f04f5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.969315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf9a91-a44c-4b2d-b900-6885669f04f5 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.970697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 396/797] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:52.976162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:52.976456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:52.976734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:52.995001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:52.996281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:53.003448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.004371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.004855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.005726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.015554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.017456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:53.031650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:53.032587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:53.033478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 402/798] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:53.033986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.053583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.061801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.100357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 397/799] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:27:53.101287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.164115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b7b47-4254-448d-9c18-779c6797a43b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.202035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.203030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.203534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.204007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.212621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.214820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:53.228301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:53.229274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:27:53.230227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:53.233408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 403/800] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:53.238935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.324425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444fdca-2834-4345-bcbb-5dd2ceac1308 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.334875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.370475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.371227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f52cf-60a2-4f63-b814-573331e33e66 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.371727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.372245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.372826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.373828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:53.374369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.376010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfa234-12b5-4f69-8206-b3005450a50a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.389646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 398/801] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:53.391471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca23bf4-2192-4ae0-98ca-e839579229bc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.391989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca23bf4-2192-4ae0-98ca-e839579229bc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.392527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca23bf4-2192-4ae0-98ca-e839579229bc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.393956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 399/802] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:53.398851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca23bf4-2192-4ae0-98ca-e839579229bc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.417030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.417638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.417917 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.418187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.418948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:53.419249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.420720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.437797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.443934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 404/803] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:53.444843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.508031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca23bf4-2192-4ae0-98ca-e839579229bc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.544450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.545113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.545394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.545667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/256b9d949c8e4734a6bce965a107c060/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.546080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559f7a-9684-4690-b535-96fda6bef6c6 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.547684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=256b9d949c8e4734a6bce965a107c060, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.549262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.573513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.579753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.580353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.580631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.580897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.581468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.582872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-7dcc6bf4-2ca6-48f9-bff7-c308689f538d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:53.593940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:53.596315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:53.597186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 405/804] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:53.597978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.597978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.597978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.614666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:53.616047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:53.636436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 400/805] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:53 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/256b9d949c8e4734a6bce965a107c060/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:53.636436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.701861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 406/806] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:53.702671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:53.798027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ad3b6-064b-45de-9085-d689afe8f6ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.810256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.855084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.856056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.856570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.856980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.879623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e64d6-f4df-474b-b21e-cd8ddb35b1c0 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:53.910454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:53.922617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:53.928336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:53.928336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:53.928336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:53.928336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:53.928336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:53.928336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:53.932277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 407/807] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:53.939063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.057829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.060190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.063584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3377cc-0b79-44c1-b99c-dd7cbb392cb9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.100670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.101642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.101803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.102168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.102866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:54.103236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.104807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.110763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 408/808] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:54.111610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.161671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:54.169632 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 401/809] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:27:54.170641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.180547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:54.226043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc28f2c-661e-4464-9859-85baf81ea278 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.264072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.265100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.265528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.265873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.272466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71346d4d-d0f9-47e6-be64-fedc0deaa493 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.277307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:54.283932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:54.285080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:54.286135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:54.291315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 409/810] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:54.292362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.310133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.315899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.315899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.315899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.315899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.315899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.394179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 402/811] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:27:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:27:54.395276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.411545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e225-e9d2-49b1-bb44-a6032c6cb74b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.440002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:54.466929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:54.509475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.510130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.510407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.510674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.532687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12cb6a-0190-4f23-9e87-02ca2500d7f7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.541436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.542937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:54.581944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:54.582942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:54.583835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:54.583835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:54.583835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:54.583835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 410/812] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:54.588930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.591296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.591973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.592206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.592475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.593139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:54.593439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.594802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca731af-c44b-4ab9-8588-d1a2ca430cbe None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.603603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 403/813] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:54.604692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36b49-7dc3-416c-b238-3761c8508381 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.605198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36b49-7dc3-416c-b238-3761c8508381 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.605198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36b49-7dc3-416c-b238-3761c8508381 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.607291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 404/814] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:54.611891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36b49-7dc3-416c-b238-3761c8508381 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.722815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd58a35-5364-4f39-a83a-9d70f6086a68 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.740728 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36b49-7dc3-416c-b238-3761c8508381 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:54.755228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:54.770777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.771776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.772528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.773246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.774202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:54.774672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.776758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22d0ec-38f7-495c-a157-333d1462153a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.779707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:54.780455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.780729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.781044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.792728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 411/815] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:54.794763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:54.795220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:54.795643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:54.798794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:54.800631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:54.816709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:54.817922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:54.818896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:54.818896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:54.818896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:54.820219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 405/816] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:54.821186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.823480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:54.824869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:54.856833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:54.908882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 412/817] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:54.910537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:54.959730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff52861-ffa2-435b-bce8-80c3f34844c5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.033096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5caf7b-91d5-4b41-a033-43368e35e29a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.040848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.041497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.041785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.042050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.057273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.058729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.071679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.072647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.073150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.073679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.074508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.075517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.077089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:55.081512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:55.082400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:55.083303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 406/818] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:55.093056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.093521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 413/819] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:55.098913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.112340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.161240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.215122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a390d-c581-49c8-a9c9-4ced5622b05c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.255723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.256728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.257224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.257936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.270810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.272347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:55.288201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:55.289211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:27:55.290135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:55.292225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 414/820] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:55.293313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.305901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97979f1c-c631-4fb4-a949-1d4067836e33 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.344234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.345139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.345250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.345481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.346177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.346479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.347906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.361808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 407/821] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:55.362829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.373875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.407976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92ded1-5906-4d6e-bb96-9ea90fe0c025 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.445377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.446352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.446828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.447337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] PATH_INFO: `/v3/services/network` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.448121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.449760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.455639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:55.459070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:55.460010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 14:27:55.460886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:55.463501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 415/822] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:55.464684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.472556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd1d4b-4147-4897-ae1e-9dfb6f771772 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.508927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.509533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.509808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.510076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.510794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.511095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.513121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.535700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 408/823] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:55.536576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.570653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c021-2ad4-401e-8bd9-a2ab6f3e4aab None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.604633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.605254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.605527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.605794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.607023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=0cb8a6f42b6e482eb109be9d2f40e64f) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.607323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.608780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.610462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.615811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 416/824] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/role_assignments?user.id=0cb8a6f42b6e482eb109be9d2f40e64f&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:55.616877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.642821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaaa92-c146-4b3b-93e3-be8ea5a8ef5d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.688732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.689446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.689683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.689964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.690605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] RBAC: Adding query filter params (name=network) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.690908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.692327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.706168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 409/825] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:55.711315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.728075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba107033-b80e-495c-b824-581cd6319c31 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.728799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.767724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.768667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.769117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.769540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.782238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.783980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:55.799491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:55.800630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:55.801520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:55.801520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:55.801520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:55.803654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 417/826] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:55.810679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.866261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:55.918219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b31f9-2357-474c-8860-4b3e676ae135 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.943845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb6551-4280-4194-88f9-1c447a003ee9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:55.978168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.979629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.980744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.981292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.982271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] RBAC: Adding query filter params (type=network) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.982787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:55.985912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:55.993605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 410/827] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:55.997633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:55.997633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:55.997920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:55.998576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:55.999283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:55.999575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:55.999675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:56.000982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:56.024170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 418/828] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:56.025443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:56.090928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:56.130421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daf25b-ce40-4747-87f7-536a980ae06e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:56.147720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568a22-2694-4978-8c0d-db4e46cdbeee None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:56.162462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:56.167166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd579-a41e-45a1-920b-ff9eadc8a70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:56.167866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd579-a41e-45a1-920b-ff9eadc8a70f None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.170924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd579-a41e-45a1-920b-ff9eadc8a70f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.170924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd579-a41e-45a1-920b-ff9eadc8a70f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.170924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd579-a41e-45a1-920b-ff9eadc8a70f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:56.171538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd579-a41e-45a1-920b-ff9eadc8a70f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:56.198255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:56.199594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.200124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.200589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.208853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:56.210752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:56.224608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:56.225642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:56.226525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:56.229035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 419/829] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:56.236230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:56.276590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 411/830] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:27:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:27:56.345423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8aa23-3342-477c-9734-17aad5edc5ae None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.345752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8aa23-3342-477c-9734-17aad5edc5ae None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.345991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8aa23-3342-477c-9734-17aad5edc5ae None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.347467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 412/831] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:56.353531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.354135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.354613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.356375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023677c-7217-4114-854d-4fb77c54b5e3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:56.376078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:56.378233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:56.399700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:56.400253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.400528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.400795 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.401475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:56.401775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:56.403763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:56.420172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 420/832] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:56.428736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:56.480336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 413/833] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 07 14:27:56.502392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:56.619853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc0e17-50d3-473c-b5fb-619e1c69d254 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:56.661995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c82ec1-2cdc-466e-8cef-a0531c3b9c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:56.662583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c82ec1-2cdc-466e-8cef-a0531c3b9c1d None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.662859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c82ec1-2cdc-466e-8cef-a0531c3b9c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.663131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c82ec1-2cdc-466e-8cef-a0531c3b9c1d None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/438dc1eb842e4106b704a0a3c2a88fc6/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.682638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c82ec1-2cdc-466e-8cef-a0531c3b9c1d None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=438dc1eb842e4106b704a0a3c2a88fc6, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:56.684229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c82ec1-2cdc-466e-8cef-a0531c3b9c1d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:56.765063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 421/834] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:27:56 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/438dc1eb842e4106b704a0a3c2a88fc6/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:27:56.790294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:56.862030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:56.880778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0e770-5cce-4d39-994b-50c4518b2c94 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.881167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0e770-5cce-4d39-994b-50c4518b2c94 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.881359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0e770-5cce-4d39-994b-50c4518b2c94 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.882974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 422/835] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:56.888622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.888904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.889180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.909660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:56.910981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:56.913454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3cc4f-7161-4aea-83e0-5a1fe215bdbd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:56.954325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:56.955829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:56.956116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:56.956419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:56.970344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:56.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:56.976916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:27:56.977766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:56.979549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 414/836] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:56.984976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.006509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 423/837] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:57.091055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c9190-e650-4ba6-879f-01e690816bb1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.107094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:57.124670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.125276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.125548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.125814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.128488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:57.128488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.128488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.137680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 415/838] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:27:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:57.143334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.152207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:57.250958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f396932-6310-4a69-b6a7-2c866065b1a2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.287638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.288303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.288582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.288846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.295875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.299537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.301049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.414018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d70c0-e40a-49ba-be53-aa41e693e079 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:57.436861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:57.437884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:57.438775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:57.438775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:57.438775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:57.438775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 424/839] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:57.445353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.458778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.459434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.459715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.460038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.460676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.462888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:57.474895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:57.475887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:27:57.477111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:57.480302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 416/840] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:57.482628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.520719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:57.540932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:57.570189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbe034-ce23-4503-b1d9-aed93258e8d0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.607906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.608839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.609327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.609745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.610635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:57.611180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.612779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.614169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725a19-af15-4fea-a4cb-6165b5391133 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.630847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 425/841] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:57.631931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.665513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.666283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.666888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.667261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.668461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:57.669027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.670834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.682523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 417/842] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:57.684641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.758133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a9fb7-9502-4552-8700-3a6b25860366 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.779499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:57.797815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.798928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.798928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.799088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.806996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.808479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:27:57.820565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:57.821681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 14:27:57.822847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:57.830311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 426/843] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:57.831252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:57.834259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0774-0fe4-4ecd-96f8-ec06f9ae3ada None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:57.845295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:57.883260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:57.884109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:57.884813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:57.885236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:57.886515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:57.887750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:57.889571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:57.904697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 418/844] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:57.961327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b419ed-cb23-4866-aac2-d38e529ff394 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:58.005788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:58.006413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.006746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.007268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.007750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:58.008125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:58.009672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621e1ee-106f-43ed-9db0-ec11070bdabc None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:58.030826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 427/845] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Sep 7 14:27:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:58.036565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:58.146522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59826836-8e8a-4aa2-b4d5-71992f74811b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:58.182401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:58.183006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.183281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.183549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.228513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:58.259342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:58.260829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:58.330176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a2c5d-f297-41ec-9f16-0795167c13de None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.330508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a2c5d-f297-41ec-9f16-0795167c13de None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.330725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a2c5d-f297-41ec-9f16-0795167c13de None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.332171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 428/846] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:58.337173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.337455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.337722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.359906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:58.361661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:58.370661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 419/847] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:27:58.377289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:58.396037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:58.459378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 429/848] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:27:58.461168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:58.493069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf1b10-6fad-484e-bae5-97bcedb5e371 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:58.528781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:58.529411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.529680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.529941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.540035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:58.541587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:27:58.556833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:58.557813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:27:58.558682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:27:58.558682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:27:58.558682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:58.558682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 420/849] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:58.566079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:58.673908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48233d-56a4-4569-bd99-ebabcb2ef3bf None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:58.710658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:58.711325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.711608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.711879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.712562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:58.712866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:58.714318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:58.727297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 430/850] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:58.732885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:58.734395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:58.769689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:58.853733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9cb99-3fb6-49b2-969c-33b08a8c3a08 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:58.868978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf7081-8d1d-43c7-9623-ac1df3173b1e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:58.894128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:58.895001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.895432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.895875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.896651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:58.896769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:58.898694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:58.910446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:58.910841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:58.914532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:27:58.915753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:58.916733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:58.916733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:58.916733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:58.916962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 431/851] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:27:58.919065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:58.929751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:58.931163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:27:58.950345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:58.951310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:27:58.952228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:27:58.956442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 421/852] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:58.957627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:59.031430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:59.042576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010794-0580-4066-95a7-b693c5588eb4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:59.079198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7bea2-fa6f-4d27-b054-2eb6110c3c06 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:59.081723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:59.082333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.082610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.082880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.083504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:59.083807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:59.085203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cafdc-732a-4bd7-a067-723f23ea47ab None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:59.095295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 432/853] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:59.095295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b723ef-a237-44dc-aa9d-34fbf6c26b7e None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.095295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b723ef-a237-44dc-aa9d-34fbf6c26b7e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.095295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b723ef-a237-44dc-aa9d-34fbf6c26b7e None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.095295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 433/854] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:27:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:27:59.098814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b723ef-a237-44dc-aa9d-34fbf6c26b7e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:59.122261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:59.122859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.123133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.123405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.124128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:59.124404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:59.125810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c112b-d10c-4a91-8c38-db1983322ad4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:59.137898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 422/855] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:59.144228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.144509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.144778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.161803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:27:59.162971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:27:59.196893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:59.209942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b723ef-a237-44dc-aa9d-34fbf6c26b7e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:59.231592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 423/856] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:27:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:27:59.232462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:59.255350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:59.256500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.257318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.257746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.258528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:59.258983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:59.261086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:59.270653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 434/857] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:27:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:27:59.357053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e6187-c56c-4c93-8c6f-21577ba3e2d2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:59.398981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:59.399645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.399914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.400213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.400952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=256b9d949c8e4734a6bce965a107c060) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:59.401257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:59.403361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:59.410745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 424/858] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:27:59 2022] GET /identity/v3/role_assignments?user.id=256b9d949c8e4734a6bce965a107c060&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:59.525307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:59.547401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:59.636163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25ace5-71e5-4b92-bbcf-3cf8cc82e856 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:59.673816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:59.674539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.674819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.675093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.686206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:27:59.691090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:27:59.692015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:27:59.692877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 435/859] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:27:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:27:59.698367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:59.735774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:59.812055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc9cca-5e6d-40b5-b196-cda365a6e567 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:27:59.846351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:27:59.847053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:27:59.847353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:27:59.847625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:27:59.848372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:27:59.848677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:27:59.850094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:27:59.859402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 425/860] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:27:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:27:59.865141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:27:59.948053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:27:59.971899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472c5c7-eb5f-4dd5-b8ae-5c3167df3e67 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:00.015482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:00.016119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.016429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.016702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.027454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:00.029045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:00.044891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:00.046616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:00.047761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:00.047761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:00.047761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:00.047761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 436/861] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:00.051921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:00.158595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:00.162672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce199b17-516e-4776-8219-0677e91cad67 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:00.203449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:00.204123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.204429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.204694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.205398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:00.205698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:00.207074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80e1ce-47cc-4ec7-b24c-c78ecf9651ac None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:00.220931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 426/862] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:00.226632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:00.231981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf10e-870e-420d-9afb-78bd82bc80cf None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.232298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf10e-870e-420d-9afb-78bd82bc80cf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.232579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf10e-870e-420d-9afb-78bd82bc80cf None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.234199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 427/863] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:00.241432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.241712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.241984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.260737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:00.262445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:00.344427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 428/864] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:00.364648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465b03b-d11e-4278-a2b2-5f58d57c1aef None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:00.398388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:00.405559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:00.406433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.406977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.407447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.417260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:00.417260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:00.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:00.429399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:00.430349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:00.432837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 437/865] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:28:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:00.439398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:00.536697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:00.550027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8ec94-878d-4dac-b3a7-192f67357dca None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:00.585793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:00.586765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.587251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.587729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.588753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:00.589312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:00.590963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:00.608853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 429/866] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Sep 7 14:28:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:00.614740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:00.616465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:00.723111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac76ed-af8d-4469-bf4e-bd95c5280386 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:00.748044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe87f7a-05d3-43c6-88b3-c287accf6ac2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:00.765855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:00.766546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.766893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.767489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.790555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:00.791176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:00.791455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:00.791727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:00.792413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:00.794300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:00.834196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:00.844604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:00.846103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:00.882245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 430/867] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:00 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:00.965122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 438/868] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:28:00.970838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:00.983566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:01.095263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b6ba3-25da-4e03-9099-9621e8fe0ca3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:01.131479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:01.132363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.132641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.132908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.144185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:01.145629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:01.161087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:01.162459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:01.163350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:01.163350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:01.163350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:01.163350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 431/869] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:01.167741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:01.261113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:01.271398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908b891-e897-4f02-b6a5-5a07775ede8c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:01.306649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:01.307303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.307578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.307847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.308510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:01.309328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:01.310688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:01.325716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 439/870] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:01.331106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:01.393740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:01.445412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d0e47-fba6-43cf-99d6-ded7cf6c273b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:01.480653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:01.481797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.482073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.482342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.489252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:01.490695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1a25be62-f306-4c4f-9332-4834927ac9d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:01.502238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:01.503142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:01.504017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 432/871] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:01.509411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:01.539258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261776e7-f97d-45a7-adef-7efdc0b6c13b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.539536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261776e7-f97d-45a7-adef-7efdc0b6c13b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.539841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261776e7-f97d-45a7-adef-7efdc0b6c13b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.541460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 433/872] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:01.547008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.547308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.547565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.566552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:01.567769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:01.619315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a2e69-6ea6-4f8e-9f43-9ac55f1410c5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:01.654623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 434/873] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:01.661489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:01.662351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.662860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.663447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.664522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:01.665047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:01.666757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8eac2-d82f-4001-9729-911ef45b6001 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:01.682260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 440/874] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:01.689558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:01.740553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:01.794947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c6ec-9795-46d7-b5ad-84823e98bbcd None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.795253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c6ec-9795-46d7-b5ad-84823e98bbcd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.795506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c6ec-9795-46d7-b5ad-84823e98bbcd None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.796989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 441/875] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:01.799116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eb3a2-ccb2-47da-8905-183dc9e53c07 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:01.801923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.802201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.802468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.820617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:01.821799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:01.826506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:01.844781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:01.845806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:01.846090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:01.846367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:01.847423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=438dc1eb842e4106b704a0a3c2a88fc6) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:01.847739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:01.849952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:01.857926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 435/876] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:01 2022] GET /identity/v3/role_assignments?user.id=438dc1eb842e4106b704a0a3c2a88fc6&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:01.917489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 442/877] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:01.941094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:02.060859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460220a0-35ba-47be-85a7-afd92acbaedd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:02.071645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:02.102308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:02.103368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:02.103962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:02.105280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:02.106482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:02.107848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:02.206573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:02.213993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 436/878] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:01 2022] POST /identity/v3/services => generated 269 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:02.330175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b77386-994b-4b13-838e-29160d61f6e8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:02.379709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:02.380637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:02.381092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:02.381518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:02.382261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:02.384626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:02.387910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:02.395164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:02.396416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:02.397453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 443/879] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:02.404475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:02.517262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43565b9-8c34-42c6-a922-28d18d771647 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:02.553041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:02.553649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:02.553924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:02.554193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:02.554824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:02.555126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:02.556511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033154a-9d74-484f-8b0a-26881aaf4836 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:02.562341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 437/880] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:02.567794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:02.673608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6a6f-f4f7-44d6-8b96-19ab47e36ee2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:02.714102 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:02.714819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:02.715634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:02.715904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:02.716558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:02.716857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:02.718233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866803bb-7b57-4679-a9ef-e6c05d285dd5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:02.723350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 444/881] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:03.207991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590669-0645-4c00-b64c-8b12e120a410 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:03.208382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590669-0645-4c00-b64c-8b12e120a410 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:03.208592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590669-0645-4c00-b64c-8b12e120a410 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:03.210190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 438/882] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:03.215579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:03.216064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:03.216173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:03.237634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:03.238741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:03.300805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:03.309802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 445/883] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:03.584795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590669-0645-4c00-b64c-8b12e120a410 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:03.619322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:03.698840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590669-0645-4c00-b64c-8b12e120a410 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:03.735985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:03.736962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:03.737468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:03.737939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:03.738726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:03.740306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:03.749592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:03.750555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:28:03.751388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:03.754142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 439/884] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:28:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:03.761211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:03.860385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df39ad6-f501-4712-a4fb-d225847d4d78 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:03.895039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:03.895632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:03.895903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:03.896185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:03.896808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:03.897105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:03.898514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:03.908398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 446/885] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:28:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:03.913235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:04.023752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:04.031079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4fae-caa9-4da8-aaa5-71888a99977a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:04.066279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:04.067247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.067810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.068330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.069151 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:04.069838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:04.071493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:04.082284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 440/886] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:28:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:04.083919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:04.144756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:04.189177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce16e3-b15e-4191-a377-0e26259e625b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:04.227493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:04.228114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.228393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.228929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.229616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] RBAC: Adding query filter params (interface=public, service_id=5e0e2ba2aa0746c0a89c4fbde9d3ffa6, region_id=RegionOne) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:04.230145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:04.231647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bc91f-237e-4b86-8175-88e1a41f0639 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:04.240498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 447/887] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:28:04 2022] GET /identity/v3/endpoints?service_id=5e0e2ba2aa0746c0a89c4fbde9d3ffa6&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:04.242689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:04.306991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaddf84-3e55-4389-8374-3ea8da3cdeeb None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.307313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaddf84-3e55-4389-8374-3ea8da3cdeeb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.307545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaddf84-3e55-4389-8374-3ea8da3cdeeb None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.309708 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 448/888] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:04.314625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02f5da-7fd4-4e3e-b77a-5c8482057f9a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.314901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02f5da-7fd4-4e3e-b77a-5c8482057f9a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.315171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02f5da-7fd4-4e3e-b77a-5c8482057f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.334488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3b02f5da-7fd4-4e3e-b77a-5c8482057f9a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:04.335874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3b02f5da-7fd4-4e3e-b77a-5c8482057f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:04.352377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35bd39-299f-4750-84ab-c48b459bb192 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:04.391434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d9dad-2afd-4f40-ae3e-afd82b32de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:04.392048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9dad-2afd-4f40-ae3e-afd82b32de4a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.392354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9dad-2afd-4f40-ae3e-afd82b32de4a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.392748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9dad-2afd-4f40-ae3e-afd82b32de4a None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.393360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d9dad-2afd-4f40-ae3e-afd82b32de4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:04.394903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d9dad-2afd-4f40-ae3e-afd82b32de4a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:04.401545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 441/889] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:28:04 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:28:04.414556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 449/890] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:04.484984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8707-f52d-49f4-b109-b263a0013584 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.485282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8707-f52d-49f4-b109-b263a0013584 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.485524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8707-f52d-49f4-b109-b263a0013584 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.487256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 442/891] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:04.493046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.493330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.493606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.514054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:04.515632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:04.569134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:04.598994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 450/892] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:04.693867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf8707-f52d-49f4-b109-b263a0013584 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:04.721057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:04.803861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf8707-f52d-49f4-b109-b263a0013584 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:04.839894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:04.840891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:04.841348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:04.841842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:04.852542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:04.858467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:04.859444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:04.860320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:04.862838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 443/893] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Sep 7 14:28:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:04.869226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:04.931912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:04.991350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81614a9-bd49-44fa-b13a-93a362d572d7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.031008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.031610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.031884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.032382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.032829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:05.033130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.034492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93ffcd-408b-4aee-8e10-ffce775fc949 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.041294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.048173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 451/894] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:28:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:05.048173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3f3b-8458-49f4-bcab-7685cc6f3d53 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.048173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3f3b-8458-49f4-bcab-7685cc6f3d53 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.048173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa3f3b-8458-49f4-bcab-7685cc6f3d53 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.048173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 452/895] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:05.050746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa3f3b-8458-49f4-bcab-7685cc6f3d53 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.059267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6664b-0936-4e1a-a910-1133ede9b4d2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.093128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.093754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.094027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.094295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.094872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.096238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.104028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4e34463e-ffe8-4e7b-9b24-b25abd838e9b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:05.105006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:05.111392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:05.118186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 444/896] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Sep 7 14:28:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:05.118978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.118978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.118978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.132640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:05.134434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:05.167639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa3f3b-8458-49f4-bcab-7685cc6f3d53 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.207394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.208041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.208352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.208626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.210656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 445/897] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:05.211497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.225025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.226754 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:05.241378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:05.242755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:05.243636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:05.243636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:05.243636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:05.245758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 453/898] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:05.251358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.311222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.344489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e27fb-0447-46c8-86ce-37e574e4fc15 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.375271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.380488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.381098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.381369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.381637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.382258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:05.382556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.383633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc160a-ddc7-42c4-8d86-9bfa71abc708 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.383910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.392063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 446/899] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:05.397305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.428760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.429617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.429899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.430166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.430824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:05.431124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.433106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.452356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 454/900] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:05.456011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.518292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1cb9-87a1-438a-a23d-66902667165a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.552257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.552861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.553136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.553410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.554022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:05.554321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.555670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.560537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 447/901] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:05.561284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.571830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.574822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2010e-3b6b-479c-84ab-922e47e3d49c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.610145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.610767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.611042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.611308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.618792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.620245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:05.631437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:05.632511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:05.633544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 455/902] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:05.634172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.655597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.675271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5ba3b-e78f-4d3a-a8ef-708e3a26cf57 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.716620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.717248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.717520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.717761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.718404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.719755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.740459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb065d-6766-4a9f-8258-c35f7d834e35 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.775125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.775711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.775997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.776305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.777049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] RBAC: Adding query filter params (interface=public, service_id=6b50c01e111d478d88af8cfdd4b41dbe, region_id=RegionOne) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:05.777874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.779233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.784257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 456/903] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/endpoints?service_id=6b50c01e111d478d88af8cfdd4b41dbe&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:05.787266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.800696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 448/904] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:05 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:05.800696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:05.884863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.908097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:05.933102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8aad43-765c-4c76-8b8b-c2c958fe4f9f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.934401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf319aa-7e2a-45e4-89cc-961f799f6893 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:05.981631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.982244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.983163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.983163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.983494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:05.983796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.985261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.987315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:05.987315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:05.987743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:05.988976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:05.989680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:05.991116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:05.999309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 449/905] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:28:06.008772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 457/906] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Sep 7 14:28:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:06.015773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:06.132402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30b0a-2f9b-42f2-bce0-6fec3b3a5e11 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:06.163477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:06.173013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:06.177624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:06.177624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:06.177624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:06.245538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:06.247108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:06.351999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 450/907] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:28:06.357393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:06.463097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bfe52-3601-4aa9-b61b-2e4fe7828fec None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:06.499368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:06.500005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:06.500315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:06.500588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:06.507402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:06.512458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:06.513912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-5c8edd5f-3bf7-4334-946e-44687fea46b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:06.529357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:06.542089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:06.543032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:06.543032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:06.543032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:06.543032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 458/908] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:06.543270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:06.568019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:06.645369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677b8bb-2a76-46d6-8de5-455caf8dc570 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:06.657428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df7906-f11f-4fff-a0fa-16042bea57a4 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:06.657740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df7906-f11f-4fff-a0fa-16042bea57a4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:06.657978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df7906-f11f-4fff-a0fa-16042bea57a4 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:06.659463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 459/909] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:06.664649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:06.664934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:06.665204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:06.681320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:06.681938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:06.682218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:06.682494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:06.683136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:06.683467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:06.684459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:06.685638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:06.688828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:06.702757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 451/910] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:06.709115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:06.772270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 460/911] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:06.830513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32744e8-4248-436d-9104-1e2334279d9e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:06.837771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:06.865429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:06.875868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:06.875868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:06.875868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:06.875868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:06.875868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:06.886903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:06.887896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:06.889213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 452/912] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:06.901206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:07.009146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e1f85-add6-468d-847f-42a175699d56 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:07.030960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:07.045013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:07.045663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:07.045884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:07.046152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:07.046859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:07.047161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:07.049042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:07.070445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:07.070573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 461/913] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:07.070573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:07.165115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f6eb-0ba2-4341-98e3-8cdde1f011a2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:07.174482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98493d-1e69-4c34-8739-8af7fb8940ea None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:07.202345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:07.202985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:07.203262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:07.203532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:07.204121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:07.205721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:07.215790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:07.217063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 14:28:07.218178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:07.220214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 453/914] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:28:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:07.222483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:07.223469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:07.224292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:07.224710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:07.225134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:07.226060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] RBAC: Adding query filter params (role.id=4be78cc2dc2a437c898c9aee129296d3, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=438dc1eb842e4106b704a0a3c2a88fc6) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:07.226581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:07.228330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:07.240313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 462/915] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:07 2022] GET /identity/v3/role_assignments?user.id=438dc1eb842e4106b704a0a3c2a88fc6&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=4be78cc2dc2a437c898c9aee129296d3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:07.241345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:07.339065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e2ef7-893c-4f3c-9041-a6bbb64e0f50 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:07.377504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:07.378127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:07.378405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:07.378676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:07.379313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:07.379614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:07.381024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:07.387535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 454/916] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:28:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:07.393579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:07.449781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:07.513166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318494ac-574c-43a1-9318-b801031d8eec None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:07.549036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:07.549649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:07.549929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:07.550197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:07.550821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:07.551120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:07.552507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7fe7-d7de-48cf-92a0-4f4aee3af8e2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:07.557738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 463/917] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Sep 7 14:28:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:07.563957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:07.644588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:07.666676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd5e2f-3a20-4b89-adac-4ba37a791615 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:07.702226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475920c-4252-419f-b68f-1dd58f33baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:07.702845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475920c-4252-419f-b68f-1dd58f33baf5 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:07.703122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475920c-4252-419f-b68f-1dd58f33baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:07.703393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475920c-4252-419f-b68f-1dd58f33baf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:07.704064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475920c-4252-419f-b68f-1dd58f33baf5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:07.705464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475920c-4252-419f-b68f-1dd58f33baf5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:07.783608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 455/918] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:28:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:08.082927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf152c-7b4b-461f-87e5-002607aa9ff5 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.083307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf152c-7b4b-461f-87e5-002607aa9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.083481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf152c-7b4b-461f-87e5-002607aa9ff5 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.085119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 464/919] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:08.090993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0edc-a153-4609-b11d-76bf1dd227ba None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.091463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0edc-a153-4609-b11d-76bf1dd227ba None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.091537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a0edc-a153-4609-b11d-76bf1dd227ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.110446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f94a0edc-a153-4609-b11d-76bf1dd227ba None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:08.111667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f94a0edc-a153-4609-b11d-76bf1dd227ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:08.191410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 456/920] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:08.315082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c8270-ef93-44ec-ae82-3baac1b62dc0 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.320829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c8270-ef93-44ec-ae82-3baac1b62dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.320829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c8270-ef93-44ec-ae82-3baac1b62dc0 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.320829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 465/921] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:08.322566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.322862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.323136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.341789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:08.342986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:08.395889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:08.412781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 457/922] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:08.461714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c8270-ef93-44ec-ae82-3baac1b62dc0 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:08.571535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c8270-ef93-44ec-ae82-3baac1b62dc0 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:08.580838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:08.606672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:08.607354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.607631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.607905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.608578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:08.610009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:08.619879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:08.621824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:08.623179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 466/923] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:08.628475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:08.733354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:08.740418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d2ee9-04e5-471a-8fad-3f0d9e6dc9b1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:08.783015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:08.783618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.783896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.784188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.784826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:08.785127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:08.786926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:08.797266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 458/924] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:08.798418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:08.858984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40923f-d775-48c9-aacf-890eb83c0a92 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:08.869000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:08.900327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:08.901243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.901671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.902121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.902846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:08.904395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:08.913878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:08.914845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 14:28:08.915717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:08.918712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 467/925] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Sep 7 14:28:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:08.924524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:08.939717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac51b4-0be5-4837-90d3-489811607619 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:08.967966 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:08.985526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:08.986153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:08.986432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:08.986702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:08.987610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:08.987922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:08.989475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:08.996333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 459/926] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:09.002096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:09.052233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689162f0-5415-4878-9cc7-fea0f3637184 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:09.093085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:09.093854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.094135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.094405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.095208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:09.095509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:09.096946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:09.106954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 468/927] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:09.107968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:09.132396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462b89e-804c-4021-bcde-5dcee788aed2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:09.149435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:09.176303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:09.177355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.177736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.178113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.179186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] RBAC: Adding query filter params (interface=public, service_id=5a52c76ea4854737ae04ffa24c90e109, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:09.179997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:09.181769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:09.188983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 460/928] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:28:09 2022] GET /identity/v3/endpoints?service_id=5a52c76ea4854737ae04ffa24c90e109&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:09.194907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:09.228326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:09.232329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3027263-7fe3-4018-905f-16606ac367ae None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:09.269860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:09.270468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.270745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.271017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.271638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:09.271937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:09.273336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:09.283517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 469/929] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:09.284837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:09.333588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f885d-dece-466a-992c-e2ade2b04a34 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:09.379708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a4b15-9e90-44d9-89fa-28ab055562dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:09.381434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a4b15-9e90-44d9-89fa-28ab055562dd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.381982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a4b15-9e90-44d9-89fa-28ab055562dd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.382683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a4b15-9e90-44d9-89fa-28ab055562dd None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.383600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a4b15-9e90-44d9-89fa-28ab055562dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:09.386067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a4b15-9e90-44d9-89fa-28ab055562dd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:09.400362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0db1b-e1e5-40f9-b9ac-6b41c9075a52 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:09.400633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 461/930] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:28:09 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:28:09.441984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:09.442603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.442878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.443144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.443886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:09.445378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:09.511158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06560b73-7f26-4ed7-b97e-678098440597 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.511433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06560b73-7f26-4ed7-b97e-678098440597 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.512326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06560b73-7f26-4ed7-b97e-678098440597 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.515152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 462/931] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:09.521575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:09.521854 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:09.522149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:09.543044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:09.544343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:09.566027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 470/932] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:28:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:09.594240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:09.639094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 463/933] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:09.906187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:09.929312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:10.016041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae3dd8-e239-4b71-9b70-07c176dc6a06 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.051257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b465771-3237-4d5f-935a-a4411fbd216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.051880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465771-3237-4d5f-935a-a4411fbd216e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.052188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465771-3237-4d5f-935a-a4411fbd216e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.052442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465771-3237-4d5f-935a-a4411fbd216e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.062658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465771-3237-4d5f-935a-a4411fbd216e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-7b465771-3237-4d5f-935a-a4411fbd216e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:10.067390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:10.068341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:10.069187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.073505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 471/934] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Sep 7 14:28:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:10.075422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.100796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ffb9f-bd49-47cd-b3d5-9581d810eb7d None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.101120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ffb9f-bd49-47cd-b3d5-9581d810eb7d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.101353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ffb9f-bd49-47cd-b3d5-9581d810eb7d None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.103506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 472/935] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:10.108630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.108920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.109182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.128000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:10.129432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:10.190454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff97af-6e75-4ff3-8a8c-04e7e55367c8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.206430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 473/936] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:10.234752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.235363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.235666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.235907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.236723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:10.237029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:10.238393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:10.247629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 464/937] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:10.253501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.267127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:10.360888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbc16c-5c61-426a-9b28-ab0b401fde19 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.399465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.400395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.400812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.401238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.411883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:10.413531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:10.428610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:10.429747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:10.430846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:10.430846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:10.430846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.437510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 474/938] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:10.438990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.472061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:10.509948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.562752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f0df0-c9f5-4c12-b163-6f3865159a5b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.583791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:10.608589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.609325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.609603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.609870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.610816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:10.611120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:10.612507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:10.627432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806b51-f4cc-44c2-9f05-480aa6e25819 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.628639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 465/939] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:10.634138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.666385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.667306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.668412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.668913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.669757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:10.671402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:10.674070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING keystone.server.flask.common [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 14:28:10.725593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.sql.core [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 14:28:10.725593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 14:28:10.725593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [parameters: {'id': 'dfcab4c57695479288e536350162bc28', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 14:28:10.725593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81804) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 14:28:10.734475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dfcab4c57695479288e536350162bc28.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dfcab4c57695479288e536350162bc28. Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application conn.query(q) Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 14:28:10.742318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result.read() Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 14:28:10.750149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application next(self.gen) Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application yield resource Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application next(self.gen) Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise self.value Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application session.commit() Sep 07 14:28:10.751334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.session.flush() Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application compat.raise_( Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 14:28:10.752509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 14:28:10.753605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application conn.query(q) Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result.read() Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 14:28:10.754560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [parameters: {'id': 'dfcab4c57695479288e536350162bc28', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.755730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:10.756874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dfcab4c57695479288e536350162bc28. Sep 07 14:28:10.757991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.758592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902dbd0-ac82-4d07-bbcf-86de2bded2f4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.758676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 475/940] 10.222.0.57 () {60 vars in 1140 bytes} [Wed Sep 7 14:28:10 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:10.758676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.803550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.804840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.805118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.805404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.815689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:10.817254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:10.829863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:10.830868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:10.831815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:10.835759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 466/941] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:10.836759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:10.890419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662b26d-19bb-4933-95cd-cb6e5615067c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.908719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:10.927098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:10.928016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:10.928473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:10.928981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:10.936746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:10.938597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:10.956659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:10.957589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:10.958463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:10.967391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3db95d-3863-46c9-961c-99c4a945a747 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:10.968050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 476/942] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:10.973110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:11.037045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:11.038057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.038528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.039008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.040008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:11.040537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:11.042202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:11.083173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 467/943] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:11.085310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:11.124562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:11.198673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b146e-84ff-4d0e-9222-1bd0d65bc009 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:11.239361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:11.242202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604e93a-bc8e-4c24-99f4-d02c7f1dc557 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:11.249086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.249410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.249684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.268208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:11.269783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:11.295710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:11.296777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:11.297673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:11.297673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:11.297673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:11.297673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 468/944] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:11.300794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:11.301635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.302112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.302566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.302672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:11.303507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] RBAC: Adding query filter params (name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:11.303992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:11.305711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:11.349724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 477/945] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:11.424510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:11.471219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463b2aa-f224-4b34-9f84-d4e164c395f0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:11.530642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:11.531615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.532089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.532835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.533679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:11.534185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:11.535770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923aaee5-2f5c-4367-84eb-8610cba3ca45 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:11.550270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 469/946] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:11.555705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:11.613161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:11.677246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c80327-57f3-419c-97c2-2c1e3501d855 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:11.712961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:11.713873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.714360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.714803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.718947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19b81-c921-477d-ba4a-e9b20944dc98 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.719234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19b81-c921-477d-ba4a-e9b20944dc98 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.719504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19b81-c921-477d-ba4a-e9b20944dc98 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.720990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 470/947] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:11.722193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:11.723643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:11.727178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea481e5-2a11-4668-a37e-b129e4d33b69 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.727460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea481e5-2a11-4668-a37e-b129e4d33b69 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.727731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea481e5-2a11-4668-a37e-b129e4d33b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:11.735433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:11.736674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:11.737558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 478/948] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:11.742874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:11.754921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-aea481e5-2a11-4668-a37e-b129e4d33b69 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:11.756375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-aea481e5-2a11-4668-a37e-b129e4d33b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:11.838105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 471/949] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:11.864991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:11.867385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d26c6-70a3-4e8a-b059-329480721dd1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.867669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d26c6-70a3-4e8a-b059-329480721dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.867942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d26c6-70a3-4e8a-b059-329480721dd1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.869606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 472/950] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:11.874731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28b47-40cc-4dd1-922a-f30af5ac8b3e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:11.875045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.875545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.876020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.894767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:11.896280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:11.910880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:11.911487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:11.911760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:11.912078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:11.912904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:11.913249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:11.915143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:11.928643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 479/951] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:11.934656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:11.957771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:11.992226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 473/952] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:12.058054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161b44-0334-4778-9c18-ff597e2fbe08 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.093992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.094570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.094850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.095118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/438dc1eb842e4106b704a0a3c2a88fc6/roles/4be78cc2dc2a437c898c9aee129296d3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.097433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=438dc1eb842e4106b704a0a3c2a88fc6, role_id=4be78cc2dc2a437c898c9aee129296d3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.098865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.117532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.168527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 480/953] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:28:11 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/438dc1eb842e4106b704a0a3c2a88fc6/roles/4be78cc2dc2a437c898c9aee129296d3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:28:12.238947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5cac6-fc2d-4820-b767-f6745ce8f195 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.261237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.280468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.281226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.281509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.281780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.282376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.283907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.292734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:12.295186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:12.296796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:12.297723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 474/954] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:12.304849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.364287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:12.409834 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d17d1-8d9b-464a-a016-f3e06702bed2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.434883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800b2b-e018-4ebd-acdb-3c0c6229693e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.458440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.459438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.459920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.460428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.461826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.463732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:12.476675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:12.477694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:12.478517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:12.479360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.479360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.479360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.479560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.480284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:12.481125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.482610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.482822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 481/955] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:12.488719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.488861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 475/956] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:12.500614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.584849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:12.604356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7076ab-2b0b-468a-bd8b-57097b4013c5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.625372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae364-eb19-4018-8f79-dbe9c81de033 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.638999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:12.642831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.644052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.644581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.645040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.645857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:12.646308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.647851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.653982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 482/957] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:12.660122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.677024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.678337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.679069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.679526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.680614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:12.681429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.682979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.690193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 476/958] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:12.695998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:12.788316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e225f-61a3-4f8f-a60f-3b2df65467f9 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.825927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71d2f-4eb6-413a-a88f-f06fd08614a2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:12.835159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.836265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.836721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.837415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.838214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:12.838648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.841087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e1d1f-e921-4ca5-b0d2-a09f1849b614 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.847572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 483/959] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:12.867395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e8c51-96b6-467c-b4ec-1a4fc93f8450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:12.868064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8c51-96b6-467c-b4ec-1a4fc93f8450 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:12.868418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8c51-96b6-467c-b4ec-1a4fc93f8450 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:12.868731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8c51-96b6-467c-b4ec-1a4fc93f8450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:12.869467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e8c51-96b6-467c-b4ec-1a4fc93f8450 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:12.870955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e8c51-96b6-467c-b4ec-1a4fc93f8450 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:12.958677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 477/960] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:28:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:13.809623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e563-b9bc-4aea-89c7-7b9f244305f9 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:13.810327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e563-b9bc-4aea-89c7-7b9f244305f9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:13.810327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e563-b9bc-4aea-89c7-7b9f244305f9 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:13.811760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 484/961] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:13.817056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553962b5-6ad2-4215-b5f2-c7feeb56dc75 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:13.817542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553962b5-6ad2-4215-b5f2-c7feeb56dc75 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:13.817700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553962b5-6ad2-4215-b5f2-c7feeb56dc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:13.836720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-553962b5-6ad2-4215-b5f2-c7feeb56dc75 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:13.837911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-553962b5-6ad2-4215-b5f2-c7feeb56dc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:13.863662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:13.920486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 478/962] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:14.189940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52e563-b9bc-4aea-89c7-7b9f244305f9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:14.202608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:14.301821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52e563-b9bc-4aea-89c7-7b9f244305f9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:14.337170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:14.337777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:14.338058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:14.338334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:14.338984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:14.341065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:14.387636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1192bc-ad97-4569-8b41-1aad350f281d None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:14.387937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1192bc-ad97-4569-8b41-1aad350f281d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:14.388223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1192bc-ad97-4569-8b41-1aad350f281d None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:14.388416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.sql.core [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 14:28:14.388416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 14:28:14.388416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [parameters: {'user_id': 'ddf1371b070541fda3940ee5e1de00d3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 14:28:14.388416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81804) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 14:28:14.390261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 479/963] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:14.395450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application conn.query(q) Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result.read() Sep 07 14:28:14.395700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 14:28:14.396595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application next(self.gen) Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application yield resource Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application next(self.gen) Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise self.value Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application session.commit() Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 14:28:14.397544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.session.flush() Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application compat.raise_( Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 14:28:14.398434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 14:28:14.399492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application conn.query(q) Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application result.read() Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 14:28:14.403084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ddf1371b070541fda3940ee5e1de00d3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:14.406253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 14:28:14.408083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 14:28:14.413128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 14:28:14.413128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 14:28:14.413128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 14:28:14.413128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 14:28:14.413128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 14:28:14.413128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.413908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:14.413908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:14.414174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 485/964] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Sep 7 14:28:14 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:14.416313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:14.417268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:14.417535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:14.491785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 480/965] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:14.509527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:14.540907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a3b1-1882-45f1-a583-f34d7e796305 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:14.575463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:14.576501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:14.579124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:14.579124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:14.585243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:14.587045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:14.600040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:14.601207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 14:28:14.602348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.604668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 486/966] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:28:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:14.610611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:14.700302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:14.716813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47d496-c6b7-4f72-950a-ea0096a64008 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:14.752408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:14.753303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:14.753743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:14.754331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:14.755485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:14.755943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:14.757529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:14.760127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:14.796093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 481/967] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Sep 7 14:28:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:14.884260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b5ed7-2a9e-40e9-901e-4044b0ade863 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:14.918031 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:14.924320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511c798-61fb-4377-91f4-485dc1729c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:14.925286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511c798-61fb-4377-91f4-485dc1729c05 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:14.935997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511c798-61fb-4377-91f4-485dc1729c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:14.935997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511c798-61fb-4377-91f4-485dc1729c05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:14.942378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511c798-61fb-4377-91f4-485dc1729c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-7511c798-61fb-4377-91f4-485dc1729c05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:14.947616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:14.949355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:14.950189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 487/968] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Sep 7 14:28:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:14.956375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.068571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52ab38-8ca1-4705-99b2-ab9f942b2135 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.086016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef3eb7-97bc-4075-970e-a06da287d432 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.086304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef3eb7-97bc-4075-970e-a06da287d432 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.086574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef3eb7-97bc-4075-970e-a06da287d432 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.091762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 488/969] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:15.097473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744925e-60d4-4fa1-b659-087de4404ed9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.097787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744925e-60d4-4fa1-b659-087de4404ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.098081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744925e-60d4-4fa1-b659-087de4404ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.110325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.111221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.111498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.111766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.113496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:15.113496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.114500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.123591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 482/970] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:28:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:15.129021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4744925e-60d4-4fa1-b659-087de4404ed9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:15.129749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.131222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4744925e-60d4-4fa1-b659-087de4404ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:15.134400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:15.208981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 489/971] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:15.226331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c920742-7dfb-4a04-90b0-da78043c6431 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.226699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c920742-7dfb-4a04-90b0-da78043c6431 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.226881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c920742-7dfb-4a04-90b0-da78043c6431 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.228261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 490/972] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:15.234083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.234368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.234645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.249750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca38ef-42ca-4dd8-b29d-31850f82826b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.253871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:15.255052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:15.287274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.287900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.288319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.288591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.300284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.301973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:15.317111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:15.318281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:15.319433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:15.319433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:15.319433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:15.322962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 483/973] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:15.324366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.326310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 491/974] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:15.383520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:15.427524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d553971-030e-4cb1-b4a2-07578b934014 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.461478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.462136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.462415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.462685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.463420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:15.463726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.465161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.477185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 484/975] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:15.477377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.482485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.532802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:15.592670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323a783-49d8-411d-9357-c4f02f1181b2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.594059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6987b9e-5f05-4b42-92e5-1c05693c6bbc None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.609984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:15.627704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.628302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.628426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.628640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.628910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.629023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.629189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.629459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.630106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.631473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.635921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.637968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:15.650244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:15.651343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 485/976] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:15.652323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.709395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 492/977] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:15 2022] POST /identity/v3/services => generated 242 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:28:15.710255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.766737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493924c6-d3f5-4a0c-94c2-1247faf10229 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.818139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.818797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.819129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.819425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.819927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.821352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.829224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8859d2-4965-4264-85f9-9f40cd431cf6 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:15.830940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:15.831892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:15.833368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 486/978] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:15.838302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.848342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:15.874831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.875614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.875895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.876508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.877570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:15.877880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.879465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:15.880050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:15.900939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 493/979] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:15.907174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:15.954649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041f6e-2fae-4664-ab38-caca32e56ec1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:15.993526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:15.994763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:15.995226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:15.995776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:15.997474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:15.997474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:15.998998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.004213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 487/980] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:16.009625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.032336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ae39f-e21b-4c90-8fb3-e6ea7f70c50d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.068858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.069470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.069749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.070021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.113196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:16.118585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:16.121792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf959c-0f31-418d-b706-ab5497cd4609 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.143020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.145196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.157738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.158355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.158628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.158895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.159573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:16.159872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.161269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.166472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 488/981] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:16.171988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.268976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 494/982] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:15 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:28:16.275402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.293040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352cb37-8e42-41f0-be4d-d186ff1c5304 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.331456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.332414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.332696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.333124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.333844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] RBAC: Adding query filter params (interface=admin, service_id=5a52c76ea4854737ae04ffa24c90e109, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:16.334155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.335519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.341298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 489/983] 10.222.0.57 () {56 vars in 1249 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/endpoints?service_id=5a52c76ea4854737ae04ffa24c90e109&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:16.346797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.365020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:16.377598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:16.397413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d8bac-40cc-4d64-a281-45b246039da0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.434146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.434779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.435080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.435351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.446359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.447823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.453718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f173c4-ff5c-454e-9091-03b46f2915d4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:16.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:16.464036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:16.465513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:16.465513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:16.465513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:16.465513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 495/984] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:16.470192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.496816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.497429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.498157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.498157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.498656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.499915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.506519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 490/985] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:28:16.583214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b4179-760a-4429-a1bc-cfc2a270c023 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.609095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:16.622366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.622366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.622641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.622891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.623469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:16.623771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.625225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.645063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 496/986] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:16.646823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.742599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:16.752448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e67d-9923-4b81-8c40-1bcdc966360a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.786869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.787840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.788368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.788840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.796368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.798024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:16.809972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:16.810857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:16.811773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:16.817400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 491/987] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:16.818726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:16.922743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304db92-7559-4c61-bb8d-bc2d730e250f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:16.956137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:16.956768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:16.957044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:16.957314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:16.958002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:16.958305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:16.959731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91c9e-f305-4764-a9ec-a241caa680fb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:16.978848 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 497/988] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:16.985484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:17.087562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7da343-ac61-455a-9042-701365d02050 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:17.122786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:17.123392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:17.123666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:17.123936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:17.124684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] RBAC: Adding query filter params (role.id=4be78cc2dc2a437c898c9aee129296d3, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=438dc1eb842e4106b704a0a3c2a88fc6) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:17.124990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:17.126403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e748ac4-553b-4609-aa62-3315d54df332 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:17.132632 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 492/989] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:16 2022] GET /identity/v3/role_assignments?user.id=438dc1eb842e4106b704a0a3c2a88fc6&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=4be78cc2dc2a437c898c9aee129296d3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:17.929900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76740-509d-494c-a55b-46530ae6d8ec None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:17.930429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76740-509d-494c-a55b-46530ae6d8ec None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:17.930429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76740-509d-494c-a55b-46530ae6d8ec None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:17.932067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 498/990] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:17.937755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:17.938043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:17.938315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:17.956784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:17.957976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:17.969961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:18.031387 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 493/991] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:18.305944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76740-509d-494c-a55b-46530ae6d8ec None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:18.356467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:18.416943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76740-509d-494c-a55b-46530ae6d8ec None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:18.451502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:18.452266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:18.452551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:18.452825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:18.453411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:18.454782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:18.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:18.464631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:18.465483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:18.466839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 499/992] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:18.470628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:18.570428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a55ef-9340-4ae6-91ae-e6ef8fb89433 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:18.609337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:18.609954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:18.610233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:18.610548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:18.611128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:18.611497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:18.612885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba41b98-02ec-4b24-8cf5-e4e2970f1983 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:18.617921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 494/993] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:18.624493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:18.698080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c18779-a8a7-4f92-9199-60c05b61019b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:18.698406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c18779-a8a7-4f92-9199-60c05b61019b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:18.698668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c18779-a8a7-4f92-9199-60c05b61019b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:18.700197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 495/994] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:18.705194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:18.705485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:18.705754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:18.724610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:18.725802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:18.728381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d2a2-1cf3-4bc9-961b-9eff4958f983 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:18.728691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:18.737568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:18.766417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:18.767020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:18.767299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:18.767569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:18.768221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:18.768532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:18.769913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:18.774861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 500/995] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:18.781468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:18.799043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 496/996] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:18.884433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86c219-dd22-4cd6-a068-21766a5dfa97 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:18.919996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:18.920646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:18.920930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:18.921202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:18.921895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] RBAC: Adding query filter params (interface=public, service_id=c8bc405f1eea442e8677e9499f6b564c, region_id=RegionOne) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:18.922208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:18.923605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:18.928939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 501/997] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:28:18 2022] GET /identity/v3/endpoints?service_id=c8bc405f1eea442e8677e9499f6b564c&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:18.934345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.046960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbae53a-3312-4e28-86fd-0a19e428dff9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:19.072138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.084448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:19.085376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.085663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.085939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.086547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:19.087907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:19.094710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 497/998] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:28:18 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:28:19.120516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:19.199584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ece8e4-d0c9-43f9-89b4-05f2146dbaff None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:19.239793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:19.240486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.240756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.241028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.241625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:19.243035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-f61c8797-0ae1-4776-b864-61ab061c7579 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:19.252755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:19.253814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 14:28:19.254642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:19.256523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 502/999] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Sep 7 14:28:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:19.259972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.264190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:19.320842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a37ff-642b-4ca7-b79d-d357b9429f61 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.321165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a37ff-642b-4ca7-b79d-d357b9429f61 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.321488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a37ff-642b-4ca7-b79d-d357b9429f61 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.323144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 503/1000] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:19.328523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.328807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.329392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.350549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:19.351769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:19.369241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3513d6-07a5-4f86-bee7-6ceb7b957772 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:19.405797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:19.406408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.406691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.406951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.407579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:19.407885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:19.409307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:19.414359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 498/1001] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:19.419736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.422101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 504/1002] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:19.486718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:19.521597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a692519-75f2-4ba7-8fa1-8e7f4c14c1eb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:19.555966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:19.556605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.556885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.557152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.557774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:19.558103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:19.559568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:19.566538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 499/1003] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Sep 7 14:28:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:19.572004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.602808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:19.676598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3305a-eaba-4476-8b18-7eeed1f79ba6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:19.696175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.712379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:19.712995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.713271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.713542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.714276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:19.716181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:19.799356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04df7-ed60-467f-9003-62bf68432660 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:19.830162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:19.840175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 505/1004] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:28:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:19.850737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:19.851379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:19.851650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:19.851912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] PATH_INFO: `/v3/services/image` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:19.852580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:19.853969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:19.863682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:19.865078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:19.865923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:19.868320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 500/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Sep 7 14:28:19 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:19.871209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:19.978093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:19.992817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1c6a7-d460-4d0c-aa2d-3bfd17533088 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:20.033149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:20.033755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:20.034040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:20.034310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:20.034940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] RBAC: Adding query filter params (name=image) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:20.035239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:20.036920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1df4-cafe-4c1d-b2b4-8aea75d75c49 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:20.043895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 506/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:20.049997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:20.150187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a951-37c0-40ff-a6dc-bcc7b1e9b950 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:20.184712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:20.185314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:20.185587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:20.185851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:20.186468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] RBAC: Adding query filter params (type=image) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:20.186766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:20.188116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be00d0d-da64-45a8-8a6a-524e2173a344 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:20.192510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 501/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:21.283910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb67a3-f167-4d67-93c1-816523c1d087 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:21.284442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb67a3-f167-4d67-93c1-816523c1d087 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:21.284442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb67a3-f167-4d67-93c1-816523c1d087 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:21.286006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 507/1008] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:21.291328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:21.291763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:21.291932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:21.309994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:21.311174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:21.362369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:21.379435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 502/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:21.645108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb67a3-f167-4d67-93c1-816523c1d087 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:21.748132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb67a3-f167-4d67-93c1-816523c1d087 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:21.782308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:21.782939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:21.783215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:21.783481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:21.784056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:21.785442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:21.794731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:21.795707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 14:28:21.796978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:21.800220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 508/1010] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:21.802083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:21.900355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d22b6-777b-4e2a-b684-143d04e2df73 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:21.935823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:21.936487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:21.936783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:21.937037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:21.937685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:21.937987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:21.939385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c56da-1609-4b8f-80ec-55c2184377c8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:21.945199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 503/1011] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:21.951571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:21.979358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:22.048883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e374223-1cdc-4991-b54d-25179e05739f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.049165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e374223-1cdc-4991-b54d-25179e05739f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.049442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e374223-1cdc-4991-b54d-25179e05739f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.050908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 504/1012] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:22.055527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f90c-10f2-4ae1-a6bb-233997f3ddc1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.055855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f90c-10f2-4ae1-a6bb-233997f3ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.056080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f90c-10f2-4ae1-a6bb-233997f3ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.062499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92d98-9ab7-4326-b5a0-8db41a66cbc1 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:22.074922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d855f90c-10f2-4ae1-a6bb-233997f3ddc1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:22.076112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d855f90c-10f2-4ae1-a6bb-233997f3ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:22.100023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:22.100815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.101232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.101510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.102144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] RBAC: Adding query filter params (type=compute) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:22.102445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:22.103812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:22.108874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 509/1013] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Sep 7 14:28:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:22.114390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:22.124360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:22.146815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 505/1014] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:22.198089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:22.218349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898783b-c4c5-43f8-a78c-968cf922a9df None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:22.255383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:22.256064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.256372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.256648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.257325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:22.258737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:22.363171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae072d-ad08-450a-a190-2be740cb3240 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.363784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae072d-ad08-450a-a190-2be740cb3240 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.363784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae072d-ad08-450a-a190-2be740cb3240 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.365207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 506/1015] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:22.370328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.370608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.370880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.389389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:22.390825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:22.455724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 507/1016] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:22.456574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:22.466586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 510/1017] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:28:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:28:22.569797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4c534-7be0-454b-92a3-ab4de7ddf5aa None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:22.582729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:22.616440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:22.617100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.617381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.617651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.618842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:22.620929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:22.622776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 508/1018] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:22.628511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:22.727713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:22.736302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7cd86-cd92-4535-a45b-9f3b585b9f87 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:22.775012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dc907-d108-4ee0-b6c3-bef49c82595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:22.775997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dc907-d108-4ee0-b6c3-bef49c82595e None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.776578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dc907-d108-4ee0-b6c3-bef49c82595e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.777572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dc907-d108-4ee0-b6c3-bef49c82595e None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.778430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dc907-d108-4ee0-b6c3-bef49c82595e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:22.780463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dc907-d108-4ee0-b6c3-bef49c82595e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:22.788482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:22.840094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1a7c6-b4ed-43fa-8bd0-cf775d36e9fd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:22.868193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 511/1019] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:22 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:22.884909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8d10a-b5e1-45c1-a067-cc4f53481f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:22.885641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8d10a-b5e1-45c1-a067-cc4f53481f97 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:22.886028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8d10a-b5e1-45c1-a067-cc4f53481f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:22.886386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8d10a-b5e1-45c1-a067-cc4f53481f97 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:22.887134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8d10a-b5e1-45c1-a067-cc4f53481f97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:22.888724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8d10a-b5e1-45c1-a067-cc4f53481f97 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:22.957964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 509/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:22 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:28:24.753759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db4bd-c489-44f0-b416-9b243b8d70ab None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:24.754299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db4bd-c489-44f0-b416-9b243b8d70ab None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:24.754299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db4bd-c489-44f0-b416-9b243b8d70ab None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:24.755694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 512/1021] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:24.761445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb434276-a57c-46d3-a598-79f79be78b6e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:24.761865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb434276-a57c-46d3-a598-79f79be78b6e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:24.762027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb434276-a57c-46d3-a598-79f79be78b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:24.780922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bb434276-a57c-46d3-a598-79f79be78b6e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:24.782118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bb434276-a57c-46d3-a598-79f79be78b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:24.811453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:24.860172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 510/1022] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:25.055843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c6c55-1db7-4b03-8333-dd4cef14d4c9 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.056235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c6c55-1db7-4b03-8333-dd4cef14d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.056406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c6c55-1db7-4b03-8333-dd4cef14d4c9 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.058533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 513/1023] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:25.064278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ec4da-dba3-4cab-b2db-e1545e5a7237 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.064572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ec4da-dba3-4cab-b2db-e1545e5a7237 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.064832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ec4da-dba3-4cab-b2db-e1545e5a7237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.083232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-997ec4da-dba3-4cab-b2db-e1545e5a7237 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:25.084429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-997ec4da-dba3-4cab-b2db-e1545e5a7237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:25.129607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c6c55-1db7-4b03-8333-dd4cef14d4c9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.153802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 511/1024] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:25.161535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32770aa-251e-4e3a-9ffb-58ee16aa44d8 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.161815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32770aa-251e-4e3a-9ffb-58ee16aa44d8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.162088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32770aa-251e-4e3a-9ffb-58ee16aa44d8 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.163453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 512/1025] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:25.168935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.169216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.169486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.187778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:25.189004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:25.242508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c6c55-1db7-4b03-8333-dd4cef14d4c9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.268477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 513/1026] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:25.277995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.279377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.279377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.279377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.289848 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:25.295378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:25.296374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:25.301236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 514/1027] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:25.302813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.327094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:25.407665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc9dac-ea26-49de-ac57-05fe178ecc28 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.428104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.449250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.449907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.450194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.450461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.451142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:25.451481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:25.452902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:25.461338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 514/1028] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:25.467229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.487370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:25.541270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e862a-875f-470b-8df2-5bc33fa525ef None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.552045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:25.570199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d03f0d-ef22-436f-a926-d0026f75dff2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.576476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.577172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.577389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.577660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.578896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:25.580421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:25.582004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 515/1029] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:25.582758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.608535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.609154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.609422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.609696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.619311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:25.623036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:25.637655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:25.638904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:25.640402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:25.640402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:25.640402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:25.640402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 515/1030] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:25.640402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.685828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f6c2-9997-473d-b28c-2d868ff6d760 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.702533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:25.720042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.720687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.720962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.721311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] PATH_INFO: `/v3/services/image` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.721826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:25.723207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:25.731684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:25.732862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:25.733709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 516/1031] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:25.734495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.746828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77729cb-b241-45a7-a030-2c6380f1b034 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.781671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.782285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.782533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.782803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.783443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:25.784875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:25.792775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:25.831988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c112d5-75b2-46eb-8b2a-1bc9aaf7b5ea None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:25.865298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:25.865895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:25.866176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:25.866447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:25.867174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:25.867488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:25.868855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:25.880254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 517/1032] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:25.881030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.903089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 516/1033] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:28:25 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:25.903839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:25.925819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:25.990498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b7070-9dcb-431b-91a8-2dca2952dcd2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.029369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.029600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73397881-0c50-438c-adc9-5705eb6af0ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.029992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.030266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.030533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.031185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] RBAC: Adding query filter params (name=image) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:26.031486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.032925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.042971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 518/1034] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:26.044781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.068113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.068788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.069069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.069345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.078004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.079645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:26.091766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:26.092772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:26.093825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 517/1035] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:28:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:26.099154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.158621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:26.164858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9989e-5410-4285-8f6b-f52e63f92573 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.168580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:26.201251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.201855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.202134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.202394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.203010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:26.203316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.204717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.209756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 519/1036] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:26.210671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598821-40f0-4398-879f-2ed2c816312b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.217988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.246559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.247514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.247945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.248467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.249297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:26.249609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.251033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.269985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 518/1037] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:26.275303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.379538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d65f-c5fe-4766-a52c-e5843b04c123 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.382331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e66988-eec8-4518-a04e-a81c3dd0e1d3 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.389252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:26.416363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.417021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.417302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.417573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.421535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.421535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.421739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.421935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.422767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] RBAC: Adding query filter params (interface=public, service_id=1b27ff2b41424e7eb59aac2a7c05c644, region_id=RegionOne) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:26.423037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.424430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.435096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 520/1038] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/endpoints?service_id=1b27ff2b41424e7eb59aac2a7c05c644&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:26.435818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.470178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:26.504672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.507008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.539505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6256e68-7494-4607-b946-87b6ea45f792 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.573812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.574410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.574688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.574956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.575552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.576958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.583600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 521/1039] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:28:26.617491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 519/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:28:26.618724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.724938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:26.732756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829773e-cf10-46db-828d-b62388cac07f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.771377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.772107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.772496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.772886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.783734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.785304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:26.800732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:26.801950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:26.803043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:26.803043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:26.803043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:26.807946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 522/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:26.809214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:26.819572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:26.913913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718602f-1dd3-42d4-a239-c18e410f7036 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:26.948982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:26.949886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:26.950323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:26.950739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:26.951563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:26.952018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:26.953664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:26.970798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 520/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:26.971934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:27.078942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9a780-7b4a-4d25-997a-5786ea036b9d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:27.119296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:27.121094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:27.121675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:27.122295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:27.132193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:27.134540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:27.151311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:27.152797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:27.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 523/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:27.161978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:27.231081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:27.265356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b92a7-bb90-4207-a738-97f100e4b393 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:27.300479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:27.301088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:27.301368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:27.301644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:27.302336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:27.302643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:27.304069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b57cc-1ca8-4a3d-b66b-23c6570ae590 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:27.321813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 521/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:27.323007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:27.335170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:27.426342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7dea6-8fe3-4b93-91aa-57fa5588b8c7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:27.464231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:27.464823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:27.465105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:27.465374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:27.466099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] RBAC: Adding query filter params (role.id=4be78cc2dc2a437c898c9aee129296d3, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=0cb8a6f42b6e482eb109be9d2f40e64f) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:27.466406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:27.467831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef9e97-e420-433c-b854-dd687b8ba388 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:27.473450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 524/1045] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:27 2022] GET /identity/v3/role_assignments?user.id=0cb8a6f42b6e482eb109be9d2f40e64f&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=4be78cc2dc2a437c898c9aee129296d3 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:28.135177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d98c32-4463-40a6-b9b7-f4ef02b0f067 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.136246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d98c32-4463-40a6-b9b7-f4ef02b0f067 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.136246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d98c32-4463-40a6-b9b7-f4ef02b0f067 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.137790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 522/1046] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:28.143392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31602705-c071-4793-bf75-01987754c746 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.143817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31602705-c071-4793-bf75-01987754c746 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.144002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31602705-c071-4793-bf75-01987754c746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.163410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-31602705-c071-4793-bf75-01987754c746 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:28.164631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-31602705-c071-4793-bf75-01987754c746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:28.233413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 525/1047] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:28.499141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d98c32-4463-40a6-b9b7-f4ef02b0f067 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:28.605020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d98c32-4463-40a6-b9b7-f4ef02b0f067 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:28.632013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:28.640556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:28.641184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.641453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.641717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.651931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:28.657004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:28.657925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:28.658753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:28.660223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 523/1048] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:28:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:28.664453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31602705-c071-4793-bf75-01987754c746 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:28.666641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:28.778122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31602705-c071-4793-bf75-01987754c746 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:28.811254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:28.811863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.812163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.812449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.813113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:28.813417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:28.814793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a6b47-0a52-442c-bb48-d27ab09c51d1 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:28.828251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 526/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:28.829824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:28.855966 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28506170-26fd-4f25-8a84-936d7b4a3377 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.856396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28506170-26fd-4f25-8a84-936d7b4a3377 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.856569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28506170-26fd-4f25-8a84-936d7b4a3377 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.857934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 527/1050] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:28.862868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.863159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.863443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.881348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:28.882540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:28.941525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b873e-8487-4b85-9d83-87c2dd309446 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:28.943485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:28.952983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 528/1051] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:28.981928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:28.982593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:28.982870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:28.983140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] PATH_INFO: `/v3/users/857b15bc6c0d40cab921cf18faec5352` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:28.997451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] RBAC: Authorizing `identity:get_user(user_id=857b15bc6c0d40cab921cf18faec5352)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:28.998887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.005270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 524/1052] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:28 2022] GET /identity/v3/users/857b15bc6c0d40cab921cf18faec5352 => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:29.006423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.117965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078c0be-2c73-4839-b391-5e164a29079e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.156665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.157335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.157585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.157958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.214901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.220813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:29.227902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.241627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.265128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:29.324761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f907e-8e19-4ac4-adaf-0c700e8aa493 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.341713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 529/1053] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:29 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:28:29.347233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.366128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.367035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.367571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.367841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.368670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.369991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:29.379341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:29.380388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:29.381264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 525/1054] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Sep 7 14:28:29 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:29.387321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.452976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:29.459569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d70b71-f851-46fd-a06d-5f13f4b27d37 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.496428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.497232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.497522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.497792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.498938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] RBAC: Authorizing `identity:get_project(project_id=eeba0f14f57d4adca3b837c18489ffe6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.500352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a2df-746a-4dd6-97a0-361516784ac2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.500458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.502208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 530/1055] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:29 2022] GET /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:29.507919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.513177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:29.543221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.544072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.544388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.544641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.545276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] RBAC: Adding query filter params (name=image) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:29.545598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.547993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.556202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 526/1056] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:29.560051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.627847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88936888-db5a-4828-802b-7153a4d943ad None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.665217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.665817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.666097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.666368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.667111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=eeba0f14f57d4adca3b837c18489ffe6, user.id=857b15bc6c0d40cab921cf18faec5352) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:29.667420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.668874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6cb2d-e53f-4676-9582-ab66fe9cd6c8 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.670003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8011c-c750-4ac4-b007-b611202bdb7a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.674963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 531/1057] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:29 2022] GET /identity/v3/role_assignments?user.id=857b15bc6c0d40cab921cf18faec5352&scope.project.id=eeba0f14f57d4adca3b837c18489ffe6&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:29.676236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a501708-93ed-4fff-bb37-88b393c019c6 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.676523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a501708-93ed-4fff-bb37-88b393c019c6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.676799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a501708-93ed-4fff-bb37-88b393c019c6 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.678512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 532/1058] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:29.686220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.687652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.687950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.709657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:29.710719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.710947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:29.711409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.711689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.711963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.712705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] RBAC: Adding query filter params (type=image) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:29.713019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.714541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.720216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 527/1059] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:29.725770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:29.735233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:29.772731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:29.797050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 533/1060] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:29.842757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2a7f8-66d6-43f3-a55c-1320ef45ae1e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:29.881750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:29.882546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:29.882910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:29.883284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:29.884455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:29.887460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:29.977170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 528/1061] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:28:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:28:30.073883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:30.185104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9c18e-3633-4c7b-8ab0-3ce6bf03ce3b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:30.226904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:30.227743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:30.228027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:30.228333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:30.241205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:30.246684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:30.247597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:30.248468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 534/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Sep 7 14:28:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:30.254055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:30.359569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:30.366979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4da66-a962-44ef-a21f-6bd5ca7061f7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:30.403246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558594-fc10-4350-9af7-a643df51385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:30.403866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558594-fc10-4350-9af7-a643df51385e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:30.404139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558594-fc10-4350-9af7-a643df51385e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:30.404429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558594-fc10-4350-9af7-a643df51385e None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:30.405092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558594-fc10-4350-9af7-a643df51385e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:30.405392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558594-fc10-4350-9af7-a643df51385e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:30.406741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558594-fc10-4350-9af7-a643df51385e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:30.420724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 529/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:28:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:30.422294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:30.442634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:30.536135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06dc81-5c35-42c6-b0db-1724f042d95d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:30.576601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:30.577652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:30.578195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:30.578679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:30.591550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:30.593378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:30.613701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:30.615040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:30.616169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:30.616169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:30.616169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:30.622690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 535/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:30.623668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558594-fc10-4350-9af7-a643df51385e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:30.736112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558594-fc10-4350-9af7-a643df51385e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:30.770184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:30.770789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:30.771070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:30.771324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:30.771969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:30.772301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:30.773665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:30.790550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 530/1065] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:30.792649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:30.862193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:30.902555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95092-90e3-4d6f-b7f2-d4cc210e8bee None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:30.940022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:30.940684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:30.940971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:30.941243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:30.949383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:30.950827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:30.961827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:30.962757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:28:30.964689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:30.969492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:30.969931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 536/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:28:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:30.988613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:31.070972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80045435-50ef-40c8-9d96-04e2e71a427e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:31.105995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:31.106615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.106895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.107165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.107890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:31.108217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:31.109651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:31.131885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 531/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:28:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:31.133083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:31.232443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca14e0-d055-4d46-8cf0-a37471429ee0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:31.257622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:31.269407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:31.270052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.270330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.270602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.281540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:31.283004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:31.297937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:31.298884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:31.299778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:31.299778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:31.299778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:31.299778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 537/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:31.306012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:31.384614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:31.412107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030cfd8-d0f5-4c2b-8b7d-5c564fd5a7fb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:31.446734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:31.447352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.447676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.447906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.449107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:31.449407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:31.450816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:31.467356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 532/1069] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:31.468625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:31.572562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2b9fb-d716-4db5-9b62-e857de90d312 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:31.606913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:31.607545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.607823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.608099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.615123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:31.616616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:31.628581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:31.629542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:31.630444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:31.632218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 538/1070] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:31.635554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:31.741542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae3eed-dec3-480e-a2e9-a6868faa03ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:31.778049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:31.778692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.778970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.779241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.779959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:31.780296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:31.781774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8447a9-9018-4795-914b-a545a9da9e38 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:31.783683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:31.801409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 533/1071] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:31.802702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:31.840976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:31.893016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0380-a8b7-42a4-b1e1-99762e822c4c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.893357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0380-a8b7-42a4-b1e1-99762e822c4c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.893572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0380-a8b7-42a4-b1e1-99762e822c4c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.895142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 534/1072] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:31.899977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acde0a-53bf-41da-80e9-2eb5b305d35f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.900292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acde0a-53bf-41da-80e9-2eb5b305d35f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.900669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acde0a-53bf-41da-80e9-2eb5b305d35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.908617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b83aae-e492-441f-95b0-699782bc9f16 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:31.919188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-02acde0a-53bf-41da-80e9-2eb5b305d35f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:31.920672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-02acde0a-53bf-41da-80e9-2eb5b305d35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:31.944797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95f47b-2bf9-4eb9-a9b9-b183f4bf0ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:31.945463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95f47b-2bf9-4eb9-a9b9-b183f4bf0ffb None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:31.945739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95f47b-2bf9-4eb9-a9b9-b183f4bf0ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:31.946008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95f47b-2bf9-4eb9-a9b9-b183f4bf0ffb None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/0cb8a6f42b6e482eb109be9d2f40e64f/roles/4be78cc2dc2a437c898c9aee129296d3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:31.949067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95f47b-2bf9-4eb9-a9b9-b183f4bf0ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=0cb8a6f42b6e482eb109be9d2f40e64f, role_id=4be78cc2dc2a437c898c9aee129296d3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:31.950515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95f47b-2bf9-4eb9-a9b9-b183f4bf0ffb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:31.996028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 535/1073] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:32.011745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 539/1074] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:28:31 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/0cb8a6f42b6e482eb109be9d2f40e64f/roles/4be78cc2dc2a437c898c9aee129296d3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:28:32.230241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2ef36-a4e7-4d66-a8e0-cf4ceca198c0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.230423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2ef36-a4e7-4d66-a8e0-cf4ceca198c0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.230698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2ef36-a4e7-4d66-a8e0-cf4ceca198c0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.232035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 536/1075] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:32.237229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.237668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.237777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.256546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:32.257745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:32.273369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2ef36-a4e7-4d66-a8e0-cf4ceca198c0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.330944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 540/1076] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:32.382296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2ef36-a4e7-4d66-a8e0-cf4ceca198c0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:32.408537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:32.419230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:32.419861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.420137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.420431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.430726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:32.435767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:32.436956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:32.438239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:32.440496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 537/1077] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:28:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:32.443004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.472700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:32.548190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4d1b-da94-4b4b-a8c9-00a2cdcee81a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:32.582661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:32.583268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.583548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.583817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.584507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:32.584810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:32.586180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:32.599037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 541/1078] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:28:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:32.599203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.600475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.697268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:32.710187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8389e1f-de8c-4b02-89ee-620988a0785d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:32.712847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606716-18ed-4525-959e-3b985fec8e30 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:32.750488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:32.751430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.751538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:32.751995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.752123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.752433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.752705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.752947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] PATH_INFO: `/v3/users/857b15bc6c0d40cab921cf18faec5352` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.754243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] RBAC: Authorizing `identity:get_user(user_id=857b15bc6c0d40cab921cf18faec5352)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:32.756032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:32.758224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 542/1079] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:32 2022] GET /identity/v3/users/857b15bc6c0d40cab921cf18faec5352 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:32.763742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.812661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:32.830040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:32.831514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:32.877330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab9e39-a5a8-499d-adc1-e3d975f49247 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:32.915198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:32.915828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:32.916103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:32.916405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:32.918233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] RBAC: Authorizing `identity:get_project(project_id=eeba0f14f57d4adca3b837c18489ffe6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:32.919670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:32.921571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 543/1080] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:32 2022] GET /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:32.927202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.940883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 538/1081] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:28:32.946930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:32.958575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:33.037509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:33.053708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d4ee6-ba56-46dc-8246-2b2d57fdf986 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:33.054432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b0e0d-12c3-489b-b17d-e2eda98e8b2d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:33.090772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:33.091335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:33.091613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:33.091884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:33.092486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2478ea3-c58e-4187-a3e1-c3159b163f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:33.093352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2478ea3-c58e-4187-a3e1-c3159b163f2e None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:33.093812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2478ea3-c58e-4187-a3e1-c3159b163f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:33.094255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2478ea3-c58e-4187-a3e1-c3159b163f2e None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6/users/857b15bc6c0d40cab921cf18faec5352/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:33.096423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2478ea3-c58e-4187-a3e1-c3159b163f2e None admin] RBAC: Authorizing `identity:create_grant(project_id=eeba0f14f57d4adca3b837c18489ffe6, user_id=857b15bc6c0d40cab921cf18faec5352, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:33.098060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2478ea3-c58e-4187-a3e1-c3159b163f2e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:33.102349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:33.103875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:33.118209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:33.119200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:33.120121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:33.120121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:33.120121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:33.120121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 539/1082] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:33.125276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:33.166695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 544/1083] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:28:32 2022] PUT /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6/users/857b15bc6c0d40cab921cf18faec5352/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:28:33.238925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3302cd-c41d-4108-81df-054043010899 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:33.277777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:33.278405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:33.278683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:33.278951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:33.279602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:33.279905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:33.281331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b81cb-3887-4ad5-a53f-d1aaf1784d34 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:33.284091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:33.304177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 540/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:34.226000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96c072-c6b4-4d7a-ab41-76df426cedb6 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:34.226521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96c072-c6b4-4d7a-ab41-76df426cedb6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:34.226521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96c072-c6b4-4d7a-ab41-76df426cedb6 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:34.227912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 545/1085] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:34.233398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:34.233816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:34.233941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:34.252095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:34.253307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:34.317894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 541/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:34.585397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96c072-c6b4-4d7a-ab41-76df426cedb6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:34.668735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:34.684879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96c072-c6b4-4d7a-ab41-76df426cedb6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:34.718024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:34.718652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:34.718926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:34.719195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:34.729016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:34.734792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:34.735689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:34.737333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 546/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Sep 7 14:28:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:34.741806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:34.840362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca4f7-3536-4113-bc9f-9030650cf86b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:34.864991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:34.873740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:34.874343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:34.874612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:34.874873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:34.875519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:34.875813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:34.877181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:34.889912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 542/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:28:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:34.891184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:34.991358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084128e3-4c21-40fd-9335-4f86ffda6f6c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.023934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.024576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.024852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.025120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.035648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.037130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.039528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:35.051948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:35.053038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:35.054444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:35.054444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:35.054444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:35.054444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 547/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:35.058871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.157503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa597c-2db7-4d7d-a397-15f6c581cc2e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.189641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.190249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.190525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.190795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.191427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:35.191725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.193110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8616f5-e821-459d-aece-a8ec366737f7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.205429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 543/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:35.211423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.313200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047ba0a-bea8-44e8-af4f-ad1b2b8e06a4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.323977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe60b39-1882-4a68-8284-4b8daf63ed3e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.324456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe60b39-1882-4a68-8284-4b8daf63ed3e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.324546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe60b39-1882-4a68-8284-4b8daf63ed3e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.325845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 544/1091] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:28:35.330685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.330963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.331233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.347104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.347720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.347993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.348289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.349394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:35.350582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:35.355752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.357217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:28:35.368446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:35.369424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:35.370377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 548/1092] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:35.375306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.394694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:35.418776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 545/1093] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:28:35.430007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:35.472092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4bfed-c85f-4b12-b7ac-0150581ee298 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.507414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.508009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.508318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.508587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.509305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:35.509604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.511544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.529228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 549/1094] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:35.534962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.629756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072aced-9d9b-444b-a584-af19ec4e7b6f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.662116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.662711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.662981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.663244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.691454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.707734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:35.728669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.730150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.793345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e5d24-b617-4bb0-a0a5-688897fd391a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.806986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:35.826331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.827003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.827277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.827540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.829507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 546/1095] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:28:35.835221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.838371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:35.843696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:35.844656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:28:35.846082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 550/1096] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:35.850777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:35.913571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:35.936128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2c1a6-3f44-46e4-9ef7-3ba8d441b6c9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.951767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9803a-3d5d-433b-bf9d-0735aeed997f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:35.969604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.970240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.970519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.970785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.980825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.982592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.985345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:35.985947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:35.986220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:35.986489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:35.987143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:35.987440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:35.988821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:35.997493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 551/1097] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:28:35.999124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:36.000353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:28:36.001466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:28:36.001466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:28:36.001466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:36.001466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 547/1098] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:28:36.005684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.005809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.045275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.108053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d7a79-9a63-4841-a88f-b4f0968607da None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.110276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11741b2-26a7-488a-8136-3b7e8a1f6de1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.138239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.142807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.143434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.143716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.143989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] PATH_INFO: `/v3/users/857b15bc6c0d40cab921cf18faec5352` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.144526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.145128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.145240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] RBAC: Authorizing `identity:get_user(user_id=857b15bc6c0d40cab921cf18faec5352)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.145403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.145670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.146303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:36.146576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.146683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.147943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.148352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 552/1099] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/users/857b15bc6c0d40cab921cf18faec5352 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:36.153930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.163809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 548/1100] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:36.164807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.253052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bd754-ca76-4e95-bb46-5a2810335821 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.255235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.260172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0a31-e327-49de-8418-9ef0eaec5694 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.287318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.287920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.288224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.288505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.294195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.294814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.295096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.295362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.301852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.303348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:28:36.314389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:36.315551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:36.316782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 549/1101] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:36.321410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.362008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.363512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.371694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.421973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dce24-2d7f-4bbf-a92c-d10bb3a60535 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.455197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.455786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.456061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.456356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.457028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:36.457323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.458734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.460443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 553/1102] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:28:36.466475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.469204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.474519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 550/1103] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:36.481006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.567842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342eaf7-5e24-4adf-9105-2179c39d2221 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.579327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7ab0c-8ed9-42f8-85f4-ca7ef3ac46f6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.589846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.601787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.602410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.602688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.602958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.604095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] RBAC: Authorizing `identity:get_project(project_id=eeba0f14f57d4adca3b837c18489ffe6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.605553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.607233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 554/1104] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:36.613231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:36.613357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.613835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.614113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.614382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.615104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] RBAC: Adding query filter params (role.id=4be78cc2dc2a437c898c9aee129296d3, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=0cb8a6f42b6e482eb109be9d2f40e64f) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:36.615411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.616860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602a87-40ee-4205-912c-f058852c2125 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.624246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 551/1105] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/role_assignments?user.id=0cb8a6f42b6e482eb109be9d2f40e64f&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=4be78cc2dc2a437c898c9aee129296d3 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:36.688831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:36.719699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3835b5-cd94-47a9-ba8c-65f40cc21d22 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:36.755227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:36.755838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:36.756113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:36.756399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:36.757122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=eeba0f14f57d4adca3b837c18489ffe6, user.id=857b15bc6c0d40cab921cf18faec5352) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:36.757427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:36.758900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ee4d9-b5df-45a1-b2f4-f4dfc3dd39a0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:36.766257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 555/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:36 2022] GET /identity/v3/role_assignments?user.id=857b15bc6c0d40cab921cf18faec5352&scope.project.id=eeba0f14f57d4adca3b837c18489ffe6&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:38.957449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8a1fe-4e5c-431c-a60c-d5dd5a02c419 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:38.957999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8a1fe-4e5c-431c-a60c-d5dd5a02c419 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:38.957999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8a1fe-4e5c-431c-a60c-d5dd5a02c419 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:38.959317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 552/1107] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:38.965119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:38.965544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:38.965644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:38.984453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:38.985633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:39.049720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 556/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:39.317397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8a1fe-4e5c-431c-a60c-d5dd5a02c419 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:39.419011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8a1fe-4e5c-431c-a60c-d5dd5a02c419 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:39.452398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113167e-56bb-42ae-ab48-e45e7c49c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:39.455153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113167e-56bb-42ae-ab48-e45e7c49c2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:39.455153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113167e-56bb-42ae-ab48-e45e7c49c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:39.455153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113167e-56bb-42ae-ab48-e45e7c49c2e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:39.455153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113167e-56bb-42ae-ab48-e45e7c49c2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:39.456170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113167e-56bb-42ae-ab48-e45e7c49c2e4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:39.457707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 553/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:39.463462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:39.527800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:39.561497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51aaaf-4323-409b-9aa1-ce6a737d9e91 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:39.595332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0424a-b782-477c-a47f-d60e719d724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:39.595865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0424a-b782-477c-a47f-d60e719d724d None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:39.596168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0424a-b782-477c-a47f-d60e719d724d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:39.596444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0424a-b782-477c-a47f-d60e719d724d None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:39.597084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0424a-b782-477c-a47f-d60e719d724d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:39.598511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0424a-b782-477c-a47f-d60e719d724d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:39.678866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 557/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:28:39 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:41.853525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89565493-0260-439a-9765-5973ab989ce0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:41.854643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89565493-0260-439a-9765-5973ab989ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:41.854643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89565493-0260-439a-9765-5973ab989ce0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:41.855963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 554/1111] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:41.861384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:41.861791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:41.861928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:41.881825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:41.883009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:41.946982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 558/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:42.214075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89565493-0260-439a-9765-5973ab989ce0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:42.257136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:42.312238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89565493-0260-439a-9765-5973ab989ce0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:42.344437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:42.345050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:42.345323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:42.345590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] PATH_INFO: `/v3/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:42.346985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] RBAC: Authorizing `identity:get_role(role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:42.348355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:42.349628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 555/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:42 2022] GET /identity/v3/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:42.355958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:42.407581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:42.454789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3b676-aac3-4b83-89c9-c9a803402bc7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:42.489246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:42.489805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:42.490084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:42.490352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] PATH_INFO: `/v3/users/786ebf0a972940b89dd23d4d4c1f63f3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:42.503137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] RBAC: Authorizing `identity:get_user(user_id=786ebf0a972940b89dd23d4d4c1f63f3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:42.504631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:42.506253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 559/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:42 2022] GET /identity/v3/users/786ebf0a972940b89dd23d4d4c1f63f3 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:42.512025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:42.608459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d565-392b-448e-b63b-83c87449062a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:42.618529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:42.642004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:42.642590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:42.642867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:42.643187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:42.708535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:42.709972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:42.804468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 556/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:28:42.810332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:42.907669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb33ac-6534-41c5-989b-30f513f4177f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:42.919084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:42.941465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0070d-d0b6-4cc0-8a16-7f4bcbbbd796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:42.942078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0070d-d0b6-4cc0-8a16-7f4bcbbbd796 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:42.942351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0070d-d0b6-4cc0-8a16-7f4bcbbbd796 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:42.942616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0070d-d0b6-4cc0-8a16-7f4bcbbbd796 None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:42.943703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0070d-d0b6-4cc0-8a16-7f4bcbbbd796 None admin] RBAC: Authorizing `identity:get_project(project_id=eeba0f14f57d4adca3b837c18489ffe6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:42.945156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0070d-d0b6-4cc0-8a16-7f4bcbbbd796 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:42.946769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 560/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:42 2022] GET /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:42.953016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:42.964273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:43.048650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af0a4f-a59f-4ed8-8ef6-7fa085f99eb1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:43.081378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:43.081916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:43.082191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:43.082457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:43.083174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] RBAC: Adding query filter params (role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=eeba0f14f57d4adca3b837c18489ffe6, user.id=786ebf0a972940b89dd23d4d4c1f63f3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:43.083480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:43.084937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf7594-4543-4859-a089-9392b1b800e3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:43.090749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 557/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:42 2022] GET /identity/v3/role_assignments?user.id=786ebf0a972940b89dd23d4d4c1f63f3&scope.project.id=eeba0f14f57d4adca3b837c18489ffe6&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:45.187367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f697ea-e603-43db-9a88-917f5e1dfe40 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:45.187905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f697ea-e603-43db-9a88-917f5e1dfe40 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:45.187905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f697ea-e603-43db-9a88-917f5e1dfe40 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:45.189283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 561/1118] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:45.194299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:45.194735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:45.194848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:45.212492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:45.213672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:45.279915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 558/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:45.548760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f697ea-e603-43db-9a88-917f5e1dfe40 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:45.647958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f697ea-e603-43db-9a88-917f5e1dfe40 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:45.680183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:45.680794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:45.681071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:45.681339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] PATH_INFO: `/v3/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:45.683235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] RBAC: Authorizing `identity:get_role(role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:45.684630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:45.685925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 562/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:45 2022] GET /identity/v3/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:45.691525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:45.694601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:45.788330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af652fbd-d556-4999-bf32-1520a32ca533 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:45.824044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:45.824870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:45.825147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:45.825430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] PATH_INFO: `/v3/users/786ebf0a972940b89dd23d4d4c1f63f3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:45.826822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] RBAC: Authorizing `identity:get_user(user_id=786ebf0a972940b89dd23d4d4c1f63f3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:45.829051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:45.830645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 559/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:45 2022] GET /identity/v3/users/786ebf0a972940b89dd23d4d4c1f63f3 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:45.836413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:45.882217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:45.934749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d73d94-1a07-46f7-87b0-d0250186fc74 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:45.968508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57526e44-9125-42fe-9480-e0af863e7d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:45.969206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526e44-9125-42fe-9480-e0af863e7d54 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:45.969392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526e44-9125-42fe-9480-e0af863e7d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:45.969663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526e44-9125-42fe-9480-e0af863e7d54 None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:45.970778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57526e44-9125-42fe-9480-e0af863e7d54 None admin] RBAC: Authorizing `identity:get_project(project_id=eeba0f14f57d4adca3b837c18489ffe6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:45.972248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57526e44-9125-42fe-9480-e0af863e7d54 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:45.973885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 563/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:45 2022] GET /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:45.979758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:46.047617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:46.075761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2f067-976b-4f7d-808a-1924af6d13bd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:46.107894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd385321-1618-4800-b142-a6fad64790a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:46.108561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd385321-1618-4800-b142-a6fad64790a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:46.108832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd385321-1618-4800-b142-a6fad64790a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:46.109097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd385321-1618-4800-b142-a6fad64790a7 None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6/users/786ebf0a972940b89dd23d4d4c1f63f3/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:46.111183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd385321-1618-4800-b142-a6fad64790a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeba0f14f57d4adca3b837c18489ffe6, user_id=786ebf0a972940b89dd23d4d4c1f63f3, role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:46.112617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd385321-1618-4800-b142-a6fad64790a7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:46.187326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 560/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:28:45 2022] PUT /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6/users/786ebf0a972940b89dd23d4d4c1f63f3/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:28:48.312352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f524-0cb4-4177-863a-d7c959206893 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:48.312894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f524-0cb4-4177-863a-d7c959206893 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:48.312894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f524-0cb4-4177-863a-d7c959206893 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:48.314233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 564/1124] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:48.319399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:48.319827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:48.319933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:48.338902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:48.340073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:48.406605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 561/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:48.671243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f524-0cb4-4177-863a-d7c959206893 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:48.769621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f524-0cb4-4177-863a-d7c959206893 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:48.782957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:48.802281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:48.803254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:48.803254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:48.803438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] PATH_INFO: `/v3/roles/c56cc2b2099d4b8ea626356bf510bc75` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:48.804806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] RBAC: Authorizing `identity:get_role(role_id=c56cc2b2099d4b8ea626356bf510bc75)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:48.806165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:48.808046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 565/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:48 2022] GET /identity/v3/roles/c56cc2b2099d4b8ea626356bf510bc75 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:48.813921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:48.872545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:48.910694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f7aa0-f16c-44f5-8f1e-fbfd7610ffc4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:48.944243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:48.944818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:48.945092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:48.945359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] PATH_INFO: `/v3/users/786ebf0a972940b89dd23d4d4c1f63f3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:48.946469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] RBAC: Authorizing `identity:get_user(user_id=786ebf0a972940b89dd23d4d4c1f63f3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:48.948428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:48.949982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 562/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:48 2022] GET /identity/v3/users/786ebf0a972940b89dd23d4d4c1f63f3 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:48.955619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:49.053872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3996-72d1-45a7-844d-1eddf63cdaba None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:49.086121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:49.086707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:49.086982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:49.087247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:49.133599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:49.152611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:49.154070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:49.249342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 566/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:28:49.255195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:49.351327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f94b-e5c8-431b-93a9-8d5bdb11614f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:49.379237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:49.388282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b5d36-6d6e-4863-845f-a3ef9f95538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:49.388993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b5d36-6d6e-4863-845f-a3ef9f95538b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:49.394517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b5d36-6d6e-4863-845f-a3ef9f95538b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:49.394517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b5d36-6d6e-4863-845f-a3ef9f95538b None admin] PATH_INFO: `/v3/projects/eeba0f14f57d4adca3b837c18489ffe6` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:49.394517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b5d36-6d6e-4863-845f-a3ef9f95538b None admin] RBAC: Authorizing `identity:get_project(project_id=eeba0f14f57d4adca3b837c18489ffe6)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:49.394517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b5d36-6d6e-4863-845f-a3ef9f95538b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:49.394517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 563/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:49 2022] GET /identity/v3/projects/eeba0f14f57d4adca3b837c18489ffe6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:49.400944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:49.482165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:49.499579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70db963-5704-4283-b6ad-b1b5aa012de2 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:49.532507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:49.533091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:49.533362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:49.533625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:49.534331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] RBAC: Adding query filter params (role.id=c56cc2b2099d4b8ea626356bf510bc75, scope.project.id=eeba0f14f57d4adca3b837c18489ffe6, user.id=786ebf0a972940b89dd23d4d4c1f63f3) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:49.534628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:49.536038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d250c6-b85c-427c-aea1-577945fab738 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:49.542120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 567/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:49 2022] GET /identity/v3/role_assignments?user.id=786ebf0a972940b89dd23d4d4c1f63f3&scope.project.id=eeba0f14f57d4adca3b837c18489ffe6&role.id=c56cc2b2099d4b8ea626356bf510bc75 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:51.708171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ab4d-5aac-49d1-b342-75a27ba1805a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:51.708700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ab4d-5aac-49d1-b342-75a27ba1805a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:51.708791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ab4d-5aac-49d1-b342-75a27ba1805a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:51.710051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 564/1131] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:51.715349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:51.715717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:51.715901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:51.733678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:51.734864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:51.800886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 568/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:52.074559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ab4d-5aac-49d1-b342-75a27ba1805a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:52.174608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ab4d-5aac-49d1-b342-75a27ba1805a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:52.207297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dc61f-5e54-49ad-b5ea-23b216652ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:52.207909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dc61f-5e54-49ad-b5ea-23b216652ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:52.208210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dc61f-5e54-49ad-b5ea-23b216652ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:52.208481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dc61f-5e54-49ad-b5ea-23b216652ff3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:52.209583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dc61f-5e54-49ad-b5ea-23b216652ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:52.210994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dc61f-5e54-49ad-b5ea-23b216652ff3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:52.212552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 565/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:52.218776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:52.292210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:52.313710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d4976-9ee3-42d8-9988-44441b914742 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:52.347553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4776995-6cc5-4477-8a0a-1129adbb1f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:52.348138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776995-6cc5-4477-8a0a-1129adbb1f78 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:52.348445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776995-6cc5-4477-8a0a-1129adbb1f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:52.348715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776995-6cc5-4477-8a0a-1129adbb1f78 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:52.349369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4776995-6cc5-4477-8a0a-1129adbb1f78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:52.350810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4776995-6cc5-4477-8a0a-1129adbb1f78 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:52.445125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 569/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:28:52 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:54.598148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f222f93-5494-449b-95a5-d4af11d988ae None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:54.598671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f222f93-5494-449b-95a5-d4af11d988ae None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:54.598671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f222f93-5494-449b-95a5-d4af11d988ae None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:54.600005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 566/1135] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:54.605901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:54.606296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:54.606456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:54.624279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:54.625469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:54.690695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 570/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:54.967860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f222f93-5494-449b-95a5-d4af11d988ae None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:55.033701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:55.067183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f222f93-5494-449b-95a5-d4af11d988ae None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:55.101194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110153f0-4dee-404a-b0d5-934d6603de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:55.101817 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110153f0-4dee-404a-b0d5-934d6603de53 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:55.102096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110153f0-4dee-404a-b0d5-934d6603de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:55.102367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110153f0-4dee-404a-b0d5-934d6603de53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:55.103493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110153f0-4dee-404a-b0d5-934d6603de53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:55.104953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110153f0-4dee-404a-b0d5-934d6603de53 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:55.106499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 567/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:28:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:55.112256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:55.168215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:55.208216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33638268-7d5b-41ba-805e-acb5536f5b4c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:55.241893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011b13f-ce1f-4662-8a4a-0806556b0d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:55.242424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011b13f-ce1f-4662-8a4a-0806556b0d81 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:55.242701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011b13f-ce1f-4662-8a4a-0806556b0d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:55.242967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011b13f-ce1f-4662-8a4a-0806556b0d81 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:55.243607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011b13f-ce1f-4662-8a4a-0806556b0d81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:55.245111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011b13f-ce1f-4662-8a4a-0806556b0d81 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:55.332771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 571/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:28:55 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:28:57.521040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99c51-4dca-4e23-b97e-7bb612cd12b1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:57.521560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99c51-4dca-4e23-b97e-7bb612cd12b1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:57.521560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99c51-4dca-4e23-b97e-7bb612cd12b1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:57.522876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 568/1139] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:28:57.528079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:57.528554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:57.528647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:57.546544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:28:57.547731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:28:57.612081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 572/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:28:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:28:57.882250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99c51-4dca-4e23-b97e-7bb612cd12b1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:57.981979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99c51-4dca-4e23-b97e-7bb612cd12b1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:58.017374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:58.017990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:58.018268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:58.018537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:58.020364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:58.028816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:28:58.033624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:28:58.034704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:28:58.035766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 569/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:28:58.043747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:58.086892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:58.139189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb373ea9-446e-4f8e-b130-bbb60f291fd7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:58.172426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:58.173016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:58.173289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:58.173557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:58.174207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:58.174502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:58.175864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:58.183576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 573/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:58.189483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:58.286881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf65db-6a93-4ca0-9bf7-4fb72647c15c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:58.320101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:58.320740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:58.321016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:58.321288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] PATH_INFO: `/v3/users/fb9132ae9fa3485c9fa3a814f729ed3f` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:58.334235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] RBAC: Authorizing `identity:get_user(user_id=fb9132ae9fa3485c9fa3a814f729ed3f)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:58.335670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:58.337280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 570/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:28:58 2022] GET /identity/v3/users/fb9132ae9fa3485c9fa3a814f729ed3f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:28:58.343444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:58.438851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708ca2c-3b56-4274-b07d-ea1c2bf4966c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:58.445339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:58.471294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:58.471878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:58.472174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:58.472453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:58.536513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:58.537945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:58.630541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 574/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:28:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:28:58.636826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:58.688393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:58.740351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9dd20-3cd8-4e9a-940c-fa8b1de0e6a8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:58.774521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570917c0-682b-4494-b4f0-de425a9c9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:58.775176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570917c0-682b-4494-b4f0-de425a9c9912 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:58.775453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570917c0-682b-4494-b4f0-de425a9c9912 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:58.775724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570917c0-682b-4494-b4f0-de425a9c9912 None admin] PATH_INFO: `/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:58.776861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570917c0-682b-4494-b4f0-de425a9c9912 None admin] RBAC: Authorizing `identity:get_project(project_id=75bf93a9a3514a21beef2906dc0a2c1f)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:58.778284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570917c0-682b-4494-b4f0-de425a9c9912 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:58.779897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 571/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:28:58 2022] GET /identity/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:28:58.786069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:28:58.794484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:28:58.883249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195e760-aa41-4929-83a7-eed652c5b363 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:28:58.917018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:28:58.917597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:28:58.917900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:28:58.918160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:28:58.918864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=75bf93a9a3514a21beef2906dc0a2c1f, user.id=fb9132ae9fa3485c9fa3a814f729ed3f) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:28:58.919166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:28:58.920615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd400ae-23e2-41f1-b183-26c84a41ce9c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:28:58.925703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 575/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:28:58 2022] GET /identity/v3/role_assignments?user.id=fb9132ae9fa3485c9fa3a814f729ed3f&scope.project.id=75bf93a9a3514a21beef2906dc0a2c1f&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:01.042199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814542-5974-4f6e-945f-52f9bad3bdda None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:01.042684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814542-5974-4f6e-945f-52f9bad3bdda None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:01.042799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814542-5974-4f6e-945f-52f9bad3bdda None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:01.044069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 572/1147] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:01.049265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:01.049681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:01.049810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:01.068257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:01.069441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:01.130267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:01.135193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 576/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:01.401374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47814542-5974-4f6e-945f-52f9bad3bdda None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:01.499695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47814542-5974-4f6e-945f-52f9bad3bdda None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:01.520139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:01.533385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:01.534000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:01.534277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:01.534544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:01.544848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:29:01.549699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:29:01.550765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:29:01.551803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 573/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:29:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:29:01.558697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:01.654204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfa83d-2401-47e5-878b-018c95dcd860 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:01.686762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:01.687344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:01.687616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:01.687881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:01.688558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:01.688852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:01.690209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:01.697946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 577/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:29:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:01.703625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:01.801605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbea2-7c66-45f2-bc1f-ea267524a953 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:01.834932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:01.835553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:01.835830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:01.836102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] PATH_INFO: `/v3/users/fb9132ae9fa3485c9fa3a814f729ed3f` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:01.837244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] RBAC: Authorizing `identity:get_user(user_id=fb9132ae9fa3485c9fa3a814f729ed3f)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:01.838696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:01.840286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 574/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:29:01 2022] GET /identity/v3/users/fb9132ae9fa3485c9fa3a814f729ed3f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:01.846034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:01.906261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:01.943840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36c93-7618-4797-9a42-5e800b3c4deb None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:01.977345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9616-05c4-4599-aa61-18762627e82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:01.977965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9616-05c4-4599-aa61-18762627e82a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:01.978797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9616-05c4-4599-aa61-18762627e82a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:01.979066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9616-05c4-4599-aa61-18762627e82a None admin] PATH_INFO: `/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:01.980192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9616-05c4-4599-aa61-18762627e82a None admin] RBAC: Authorizing `identity:get_project(project_id=75bf93a9a3514a21beef2906dc0a2c1f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:01.981638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9616-05c4-4599-aa61-18762627e82a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:01.983265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 578/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:29:01 2022] GET /identity/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:01.989142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:02.037724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:02.086327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a4b37-d3ee-4f76-9f63-81abe62fbefd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:02.119881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b98fd-a488-4230-ae26-23a52d7a9d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:02.120549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b98fd-a488-4230-ae26-23a52d7a9d42 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:02.120826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b98fd-a488-4230-ae26-23a52d7a9d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:02.121092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b98fd-a488-4230-ae26-23a52d7a9d42 None admin] PATH_INFO: `/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f/users/fb9132ae9fa3485c9fa3a814f729ed3f/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:02.122960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b98fd-a488-4230-ae26-23a52d7a9d42 None admin] RBAC: Authorizing `identity:create_grant(project_id=75bf93a9a3514a21beef2906dc0a2c1f, user_id=fb9132ae9fa3485c9fa3a814f729ed3f, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:02.124401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b98fd-a488-4230-ae26-23a52d7a9d42 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:02.213446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 575/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:29:01 2022] PUT /identity/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f/users/fb9132ae9fa3485c9fa3a814f729ed3f/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:29:04.350811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a989-1282-41dc-afc3-384792d73bb9 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:04.351334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a989-1282-41dc-afc3-384792d73bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:04.351334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a989-1282-41dc-afc3-384792d73bb9 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:04.352794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 579/1154] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:04.358495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:04.358902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:04.359038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:04.377791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:04.378987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:04.444578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 576/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:04.714696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992a989-1282-41dc-afc3-384792d73bb9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:04.812400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992a989-1282-41dc-afc3-384792d73bb9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:04.840875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:04.846405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:04.847023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:04.852093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:04.852093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:04.857640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:29:04.862842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:29:04.863950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:29:04.864989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 580/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:29:04.870094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:04.909313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:04.967759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0822-0479-4f86-b2de-3b402d6b669b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:05.001064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:05.001657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:05.001933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:05.002201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:05.002859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:05.003159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:05.004543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:05.012738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 577/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:05.018854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:05.124858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa202e5-29a1-4fb2-9a5e-172d83f4005c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:05.158565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:05.159180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:05.159462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:05.159731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] PATH_INFO: `/v3/users/fb9132ae9fa3485c9fa3a814f729ed3f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:05.160887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] RBAC: Authorizing `identity:get_user(user_id=fb9132ae9fa3485c9fa3a814f729ed3f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:05.162321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:05.163884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 581/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:29:05 2022] GET /identity/v3/users/fb9132ae9fa3485c9fa3a814f729ed3f => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:05.170376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:05.264909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:05.267645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e5e1a-013c-45f3-b071-83b09c0b8c17 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:05.300650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:05.301239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:05.301511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:05.301777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:05.366065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:05.367488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:05.457213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:05.460668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 578/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:29:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:29:05.467196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:05.525565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:05.565694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21dc3c-5a26-4183-b258-3be304e7322f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:05.598554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f07db-9400-49a6-9b99-8b393decfc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:05.599168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f07db-9400-49a6-9b99-8b393decfc01 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:05.599449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f07db-9400-49a6-9b99-8b393decfc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:05.599720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f07db-9400-49a6-9b99-8b393decfc01 None admin] PATH_INFO: `/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:05.600838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f07db-9400-49a6-9b99-8b393decfc01 None admin] RBAC: Authorizing `identity:get_project(project_id=75bf93a9a3514a21beef2906dc0a2c1f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:05.602279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f07db-9400-49a6-9b99-8b393decfc01 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:05.603912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 582/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:29:05 2022] GET /identity/v3/projects/75bf93a9a3514a21beef2906dc0a2c1f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:05.609886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:05.705352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd150c7-701d-4d93-867f-923dd879c2b8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:05.737083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:05.737661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:05.737932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:05.738197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:05.738901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=75bf93a9a3514a21beef2906dc0a2c1f, user.id=fb9132ae9fa3485c9fa3a814f729ed3f) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:05.739202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:05.740637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced8b20-dd8a-4401-bbbd-78d182d57986 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:05.746579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 579/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:29:05 2022] GET /identity/v3/role_assignments?user.id=fb9132ae9fa3485c9fa3a814f729ed3f&scope.project.id=75bf93a9a3514a21beef2906dc0a2c1f&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:07.892650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519568-cbf2-4665-b1da-53102f44695b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:07.893235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519568-cbf2-4665-b1da-53102f44695b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:07.893235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d519568-cbf2-4665-b1da-53102f44695b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:07.894512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 583/1162] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:07.900129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:07.900555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:07.900696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:07.918863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:07.920029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:07.977028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:07.985084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 580/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:08.249930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519568-cbf2-4665-b1da-53102f44695b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:08.350401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d519568-cbf2-4665-b1da-53102f44695b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:08.375807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:08.384181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:08.389103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:08.389103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:08.389103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:08.451084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:08.452565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:08.547403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 584/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:29:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:29:08.553281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:08.649296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9a96f-37d8-4413-9ea6-bb6ad8fbd387 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:08.682947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:08.683565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:08.683844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:08.684139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:08.690588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:08.692012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8a158186-f036-4840-a7b7-6b5dc9bbf889 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:29:08.703706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:29:08.704980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:29:08.706104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 581/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:29:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:29:08.710583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:08.736314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:08.806794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7423e-7e3e-4524-b5fa-83fd90838d89 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:08.839862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:08.840494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:08.840774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:08.841045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:08.841680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:08.841982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:08.843348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f6c0-4f81-4d17-98f5-a30b0e04b880 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:08.850259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 585/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Sep 7 14:29:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:10.948878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edf54c-fe87-49c5-b537-a3107b766319 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:10.949433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edf54c-fe87-49c5-b537-a3107b766319 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:10.949433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edf54c-fe87-49c5-b537-a3107b766319 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:10.950730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 582/1167] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:10.955956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d66e-00b3-4d36-a8df-052a15775828 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:10.956373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d66e-00b3-4d36-a8df-052a15775828 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:10.956537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d66e-00b3-4d36-a8df-052a15775828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:10.974753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d455d66e-00b3-4d36-a8df-052a15775828 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:10.975938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d455d66e-00b3-4d36-a8df-052a15775828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:11.040580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 586/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:11.305058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edf54c-fe87-49c5-b537-a3107b766319 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:11.361645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:11.406498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edf54c-fe87-49c5-b537-a3107b766319 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:11.439233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7115c-7907-4b85-b15c-b5b5080e918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:11.439819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7115c-7907-4b85-b15c-b5b5080e918e None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:11.440090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7115c-7907-4b85-b15c-b5b5080e918e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:11.440381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7115c-7907-4b85-b15c-b5b5080e918e None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:11.441041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7115c-7907-4b85-b15c-b5b5080e918e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:11.442378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7115c-7907-4b85-b15c-b5b5080e918e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:11.504928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 583/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Wed Sep 7 14:29:11 2022] POST /identity/v3/domains => generated 267 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:29:13.650392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfc99f-23b0-4095-9d6a-d9f9c54cf46d None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:13.650954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfc99f-23b0-4095-9d6a-d9f9c54cf46d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:13.650954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfc99f-23b0-4095-9d6a-d9f9c54cf46d None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:13.652344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 587/1170] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:13.657445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:13.657826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:13.657984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:13.676178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:13.677349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:13.740481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 584/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:14.005301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfc99f-23b0-4095-9d6a-d9f9c54cf46d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:14.027598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:14.107178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfc99f-23b0-4095-9d6a-d9f9c54cf46d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:14.142268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3672521-7cdf-4d9c-bb86-8ba7b0011484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:14.142985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3672521-7cdf-4d9c-bb86-8ba7b0011484 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:14.143272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3672521-7cdf-4d9c-bb86-8ba7b0011484 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:14.143545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3672521-7cdf-4d9c-bb86-8ba7b0011484 None admin] PATH_INFO: `/v3/domains/30f621ca7ad54e85801e22768ac1fac6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:14.153668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3672521-7cdf-4d9c-bb86-8ba7b0011484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=30f621ca7ad54e85801e22768ac1fac6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:14.155113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3672521-7cdf-4d9c-bb86-8ba7b0011484 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:14.156721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 588/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Wed Sep 7 14:29:14 2022] GET /identity/v3/domains/30f621ca7ad54e85801e22768ac1fac6 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:14.162541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:14.206474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:14.258352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb92de-0544-4e89-af8e-e5b8521d4973 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:14.292670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567805-2cb7-4d45-9237-acec0f086d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:14.293265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567805-2cb7-4d45-9237-acec0f086d09 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:14.293542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567805-2cb7-4d45-9237-acec0f086d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:14.293815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567805-2cb7-4d45-9237-acec0f086d09 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:14.294549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e567805-2cb7-4d45-9237-acec0f086d09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:14.295971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e567805-2cb7-4d45-9237-acec0f086d09 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:14.433874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 585/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:29:14 2022] POST /identity/v3/projects => generated 361 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:29:16.695527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8376c0e-97ed-4776-b18f-7e62ec4bfcfd None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:16.696081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8376c0e-97ed-4776-b18f-7e62ec4bfcfd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:16.696081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8376c0e-97ed-4776-b18f-7e62ec4bfcfd None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:16.697453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 589/1174] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:16.702611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:16.703057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:16.703156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:16.720891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:16.722074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:16.786474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 586/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:17.057878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8376c0e-97ed-4776-b18f-7e62ec4bfcfd None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:17.137118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:17.156589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8376c0e-97ed-4776-b18f-7e62ec4bfcfd None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:17.193126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d1de-ed85-4355-8bb5-d5ddced4e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:17.193750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d1de-ed85-4355-8bb5-d5ddced4e9cb None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:17.194026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d1de-ed85-4355-8bb5-d5ddced4e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:17.194298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d1de-ed85-4355-8bb5-d5ddced4e9cb None admin] PATH_INFO: `/v3/domains/30f621ca7ad54e85801e22768ac1fac6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:17.195418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61d1de-ed85-4355-8bb5-d5ddced4e9cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=30f621ca7ad54e85801e22768ac1fac6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:17.196903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61d1de-ed85-4355-8bb5-d5ddced4e9cb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:17.198507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 590/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Wed Sep 7 14:29:17 2022] GET /identity/v3/domains/30f621ca7ad54e85801e22768ac1fac6 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:17.204252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:17.231079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:17.301305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef20670-bca8-4b4b-a767-adae7de193e9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:17.334349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcade8-73ff-482e-86a1-ca05cce0873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:17.334953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcade8-73ff-482e-86a1-ca05cce0873b None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:17.335233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcade8-73ff-482e-86a1-ca05cce0873b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:17.335504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcade8-73ff-482e-86a1-ca05cce0873b None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:17.336176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcade8-73ff-482e-86a1-ca05cce0873b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:17.338146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcade8-73ff-482e-86a1-ca05cce0873b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:17.414652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 587/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:29:17 2022] POST /identity/v3/users => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:29:19.610860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e251-3a78-4e1b-8155-7ee810452007 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:19.611412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e251-3a78-4e1b-8155-7ee810452007 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:19.611412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e251-3a78-4e1b-8155-7ee810452007 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:19.612783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 591/1178] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:19.618113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:19.619041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:19.619294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:19.637930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:19.639121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:19.705187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 588/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:19.971957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02e251-3a78-4e1b-8155-7ee810452007 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:20.069810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02e251-3a78-4e1b-8155-7ee810452007 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:20.102833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:20.103439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:20.103710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:20.103972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:20.113669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:29:20.118455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:29:20.119542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:29:20.120566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 592/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:29:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:29:20.125411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:20.139080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:20.222539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22382e8-db6b-49bb-ae5f-ca919f85b13e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:20.255782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:20.256409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:20.256686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:20.256955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:20.257607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:20.257904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:20.259295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:20.267217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 589/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:29:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:20.273287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:20.285208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:20.373152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a3f3-b602-42f6-81ac-b97676fc079f None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:20.407355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:20.407966 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:20.408270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:20.408542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] PATH_INFO: `/v3/users/d2044a95940d45d5bee92f05d6ddd085` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:20.421391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] RBAC: Authorizing `identity:get_user(user_id=d2044a95940d45d5bee92f05d6ddd085)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:20.422876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:20.424497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 593/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:29:20 2022] GET /identity/v3/users/d2044a95940d45d5bee92f05d6ddd085 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:20.430205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:20.502029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:20.526375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c81d3-28ef-407f-a04c-bff87d98ca38 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:20.558404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:20.558997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:20.559269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:20.559538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:20.624323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:20.625753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:20.695419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:20.720132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 590/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:29:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:29:20.726240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:20.792267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:20.824766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ebac-16a6-468b-a763-5a95b05fa2d7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:20.858455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a64572-c403-4697-931a-97b981870e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:20.859070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64572-c403-4697-931a-97b981870e01 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:20.859345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64572-c403-4697-931a-97b981870e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:20.859613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64572-c403-4697-931a-97b981870e01 None admin] PATH_INFO: `/v3/projects/72ee78ecf19146a3bc01c347e4257418` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:20.860724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a64572-c403-4697-931a-97b981870e01 None admin] RBAC: Authorizing `identity:get_project(project_id=72ee78ecf19146a3bc01c347e4257418)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:20.862153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a64572-c403-4697-931a-97b981870e01 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:20.863757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 594/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:29:20 2022] GET /identity/v3/projects/72ee78ecf19146a3bc01c347e4257418 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:20.870072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:20.966379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a4667-72bd-495b-91e5-13a7a4efecea None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:20.999895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:21.000586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:21.000864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:21.001135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:21.001851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=72ee78ecf19146a3bc01c347e4257418, user.id=d2044a95940d45d5bee92f05d6ddd085) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:21.002157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:21.003570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f744f-afea-4b2f-babc-c588098bc164 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:21.009410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 591/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:29:20 2022] GET /identity/v3/role_assignments?user.id=d2044a95940d45d5bee92f05d6ddd085&scope.project.id=72ee78ecf19146a3bc01c347e4257418&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:23.126110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d987d-d3d9-4a18-9a4a-4737b4561836 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:23.126658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d987d-d3d9-4a18-9a4a-4737b4561836 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:23.126658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d987d-d3d9-4a18-9a4a-4737b4561836 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:23.127986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 595/1186] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:23.133087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:23.134071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:23.134193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:23.152273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:23.153465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:23.182210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:23.218528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 592/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:23.486729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d987d-d3d9-4a18-9a4a-4737b4561836 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:23.585629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d987d-d3d9-4a18-9a4a-4737b4561836 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:23.620023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:23.620661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:23.620942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:23.621214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:23.623007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:23.631414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:29:23.636188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:29:23.637340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:29:23.638327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 596/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:29:23.643668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:23.740767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef082b-4868-4e47-902a-77d33620e89f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:23.774751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:23.775342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:23.775623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:23.775896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:23.776580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:23.776889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:23.778244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:23.786005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 593/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:23.792234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:23.893084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d1746-97ee-43ea-8272-3d0860493491 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:23.925200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:23.925804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:23.926078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:23.926343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] PATH_INFO: `/v3/users/d2044a95940d45d5bee92f05d6ddd085` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:23.927432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] RBAC: Authorizing `identity:get_user(user_id=d2044a95940d45d5bee92f05d6ddd085)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:23.928882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:23.930934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 597/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:29:23 2022] GET /identity/v3/users/d2044a95940d45d5bee92f05d6ddd085 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:23.936718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:23.968274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:24.035749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026f63-4476-4083-93a7-2f86441a9ec8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:24.068846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e3742-0488-4ed3-8d09-1e95f3246075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:24.069458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3742-0488-4ed3-8d09-1e95f3246075 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:24.069729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3742-0488-4ed3-8d09-1e95f3246075 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:24.069990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3742-0488-4ed3-8d09-1e95f3246075 None admin] PATH_INFO: `/v3/projects/72ee78ecf19146a3bc01c347e4257418` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:24.071093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e3742-0488-4ed3-8d09-1e95f3246075 None admin] RBAC: Authorizing `identity:get_project(project_id=72ee78ecf19146a3bc01c347e4257418)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:24.072517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e3742-0488-4ed3-8d09-1e95f3246075 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:24.074096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 594/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:29:23 2022] GET /identity/v3/projects/72ee78ecf19146a3bc01c347e4257418 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:24.079978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:24.139143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:24.176091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df097eed-59f0-4f54-bb47-4bcd77bcdc7a None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:24.208747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a985fa-a3dd-4239-9b3b-fa7714cf52ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:24.209389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a985fa-a3dd-4239-9b3b-fa7714cf52ae None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:24.209664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a985fa-a3dd-4239-9b3b-fa7714cf52ae None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:24.209929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a985fa-a3dd-4239-9b3b-fa7714cf52ae None admin] PATH_INFO: `/v3/projects/72ee78ecf19146a3bc01c347e4257418/users/d2044a95940d45d5bee92f05d6ddd085/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:24.211816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a985fa-a3dd-4239-9b3b-fa7714cf52ae None admin] RBAC: Authorizing `identity:create_grant(project_id=72ee78ecf19146a3bc01c347e4257418, user_id=d2044a95940d45d5bee92f05d6ddd085, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:24.213270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a985fa-a3dd-4239-9b3b-fa7714cf52ae None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:24.280952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 598/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:29:24 2022] PUT /identity/v3/projects/72ee78ecf19146a3bc01c347e4257418/users/d2044a95940d45d5bee92f05d6ddd085/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:29:26.391989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d7232-258e-4419-a52c-ef91643fde79 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:26.392565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d7232-258e-4419-a52c-ef91643fde79 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:26.392565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d7232-258e-4419-a52c-ef91643fde79 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:26.393896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 595/1193] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:29:26.399115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:26.399534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:26.399647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:26.417720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:29:26.418892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:29:26.481930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 599/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:29:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:29:26.752785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d7232-258e-4419-a52c-ef91643fde79 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:26.847437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:26.852026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d7232-258e-4419-a52c-ef91643fde79 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:26.885461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:26.886087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:26.886364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:26.886635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:26.897204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:29:26.902606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:29:26.903732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:29:26.904645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 596/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:29:26.909645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:26.959590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:27.006349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5196be-9b3c-4d68-85b4-ab2e78e8a9a5 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:27.040437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:27.041030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:27.041304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:27.041568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:27.042221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:27.042519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:27.043880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:27.051680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 600/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:27.057821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:27.154708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2343afd-510c-48ca-a246-62e15dc01ad2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:27.186915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:27.187530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:27.187810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:27.188085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] PATH_INFO: `/v3/users/d2044a95940d45d5bee92f05d6ddd085` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:27.189220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=d2044a95940d45d5bee92f05d6ddd085)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:27.190640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:27.192193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 597/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Sep 7 14:29:27 2022] GET /identity/v3/users/d2044a95940d45d5bee92f05d6ddd085 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:27.198315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:27.295289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4325ee-83fe-4abf-89b7-000c9efb9247 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:27.308363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:27.327894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:27.328514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:27.328789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:27.329058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:27.394548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:27.395986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:27.490463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 601/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:29:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:29:27.496921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:27.528600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:27.599171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d623b0e-a339-46fc-86c5-024bcd2f6fb6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:27.632547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2c7b3-9be1-4891-a754-8e28b348563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:27.633228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2c7b3-9be1-4891-a754-8e28b348563a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:27.633988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2c7b3-9be1-4891-a754-8e28b348563a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:27.634260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2c7b3-9be1-4891-a754-8e28b348563a None admin] PATH_INFO: `/v3/projects/72ee78ecf19146a3bc01c347e4257418` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:27.635371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2c7b3-9be1-4891-a754-8e28b348563a None admin] RBAC: Authorizing `identity:get_project(project_id=72ee78ecf19146a3bc01c347e4257418)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:27.636885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2c7b3-9be1-4891-a754-8e28b348563a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:27.638491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 598/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Sep 7 14:29:27 2022] GET /identity/v3/projects/72ee78ecf19146a3bc01c347e4257418 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:27.644880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:29:27.659703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:27.742212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f7587-a80d-4400-839d-ef3ac3c63766 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:29:27.775160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:29:27.775743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:29:27.776019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:29:27.776316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:29:27.777025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=72ee78ecf19146a3bc01c347e4257418, user.id=d2044a95940d45d5bee92f05d6ddd085) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:29:27.777329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:29:27.778743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8da0b9-f44b-42b5-8598-d2a99b1af453 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:29:27.784318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 602/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:29:27 2022] GET /identity/v3/role_assignments?user.id=d2044a95940d45d5bee92f05d6ddd085&scope.project.id=72ee78ecf19146a3bc01c347e4257418&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:29:37.628420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=53 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:29:37.781780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=61 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:41.965836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda6ba-b4ff-47e4-a172-c3074c3b6c75 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:41.966405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda6ba-b4ff-47e4-a172-c3074c3b6c75 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:41.966405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda6ba-b4ff-47e4-a172-c3074c3b6c75 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:41.967707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 599/1201] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:30:41.973109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:41.973556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:41.973655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:41.992677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:30:41.993865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:30:42.061133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 603/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:30:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:30:42.341960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deda6ba-b4ff-47e4-a172-c3074c3b6c75 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:42.449116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deda6ba-b4ff-47e4-a172-c3074c3b6c75 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:42.483060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:42.488649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:42.488649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:42.488649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:42.494862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:42.496523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:42.512537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:42.518721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:42.519900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:42.519900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:42.519900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:42.519900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 600/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:42.520384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:42.623195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ca542-9273-46fd-839c-383e7467d2b9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:42.652515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:42.658463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:42.659124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:42.659430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:42.659700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:42.660361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:42.660660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:42.662069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169faf4f-dd51-49ec-9bf3-f9d00238a522 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:42.673278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 604/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:42.679557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:42.735927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:42.784018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e29af9-2c12-4361-a245-35bcaba68ca2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:42.818095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d49a08-8316-480c-88d6-a01af2e73181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:42.819116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d49a08-8316-480c-88d6-a01af2e73181 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:42.819900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d49a08-8316-480c-88d6-a01af2e73181 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:42.820242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d49a08-8316-480c-88d6-a01af2e73181 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:42.820900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d49a08-8316-480c-88d6-a01af2e73181 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:42.822444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d49a08-8316-480c-88d6-a01af2e73181 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:43.014915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 601/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Sep 7 14:30:42 2022] POST /identity/v3/users => generated 251 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:30:45.184553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b11d2-3a49-4199-919c-9bd5c87194aa None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:45.185077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b11d2-3a49-4199-919c-9bd5c87194aa None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:45.185077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b11d2-3a49-4199-919c-9bd5c87194aa None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:45.186449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 605/1206] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:30:45.191685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:45.192078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:45.192262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:45.211188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:30:45.212391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:30:45.278917 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 602/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:30:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:30:45.549427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b11d2-3a49-4199-919c-9bd5c87194aa None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:45.652380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b11d2-3a49-4199-919c-9bd5c87194aa None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:45.686320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:45.686947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:45.687228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:45.687520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:45.698158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:30:45.702960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:45.706563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:45.707629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 606/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:30:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:45.710346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:45.752545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:45.813647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc2ed4-7f2f-41f8-9336-5360cf139e05 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:45.847791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:45.848426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:45.848702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:45.849480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:45.850136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:45.850430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:45.851814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:45.859802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 603/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:30:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:30:45.866404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:45.919293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:45.973821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29183c89-7ee9-4889-acaf-018a4bd0f14e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:46.013303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:46.013946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:46.014479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:46.014778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:46.025052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:46.026492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:46.041232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:46.043560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:46.044659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:46.044659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:46.044659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:46.044659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 607/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:46.048397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:46.143484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:46.146817 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d0537-9b7e-47a3-94c5-87a389da9f34 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:46.179016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:46.179605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:46.179877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:46.180159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:46.180812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:46.181110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:46.182447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:46.193459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 604/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:46.198932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:46.297674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70e9cb-592f-4119-bdff-89041c60db5d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:46.299178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:46.331353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:46.331967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:46.332272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:46.332545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:46.339916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:46.341379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:30:46.353700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:46.354812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:46.355765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 608/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:46.361554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:46.460154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecec4-ce3b-4c13-96b1-ecd7d670558d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:46.492280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:46.492891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:46.493145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:46.493410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:46.494140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:46.494411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:46.495812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:46.512920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 605/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Sep 7 14:30:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:46.519167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:46.618523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e35cd0-40d0-4e3e-aa3d-21125247986b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:46.653337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:46.653939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:46.654220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:46.654493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:46.683503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:46.720859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:46.722244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:46.820481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 609/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:30:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 14:30:46.826464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:46.845655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:46.924340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995b9b2-2bf6-494d-a75b-018f3cc01949 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:46.958480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:46.959111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:46.959389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:46.959658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:46.969797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:46.971302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:46.985699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:46.986801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:46.988088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:46.988088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:46.988088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:46.988088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 606/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:46.993098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:47.053348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:47.091059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888a4d4-4d86-40a6-a3e3-ed43388f07b7 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:47.123801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:47.124430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:47.124711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:47.124979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:47.125614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:47.125915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:47.127270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:47.139560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 610/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:47.145346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:47.213044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:47.243324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caef91-98ca-48da-8562-fc8b4824951b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:47.275330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:47.277929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:47.277929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:47.277929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:47.283031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:47.284541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:30:47.296171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:47.297193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:47.298126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 607/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:47.303777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:47.403448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2288d-7754-40cd-8baa-650e649a6cfb None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:47.426841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:47.438887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:47.439486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:47.439762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:47.440029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:47.440754 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:47.441056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:47.442489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f5006-7bba-40d8-af34-3e9d5fe3002b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:47.453756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 611/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:47.460027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:47.561459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51e1f-f5d4-4fde-a040-17051f364d2d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:47.589040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:47.595505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:47.596157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:47.596450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:47.596723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:47.597442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=53d54b01ab3c4003b5f13e20c0849db1) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:47.597750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:47.599685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbec14-ad9f-4e86-b31b-580544049b8e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:47.605261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 608/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:30:47 2022] GET /identity/v3/role_assignments?user.id=53d54b01ab3c4003b5f13e20c0849db1&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:49.721798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f7bfd-582a-48ee-bcdd-4006934a2da2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:49.722351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f7bfd-582a-48ee-bcdd-4006934a2da2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:49.722351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f7bfd-582a-48ee-bcdd-4006934a2da2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:49.723696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 612/1220] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:30:49.728960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:49.729375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:49.736132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:49.747550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:30:49.748766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:30:49.819935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 609/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:30:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:30:50.090106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f7bfd-582a-48ee-bcdd-4006934a2da2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:50.190725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f7bfd-582a-48ee-bcdd-4006934a2da2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:50.225599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:50.226216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:50.226495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:50.226767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:50.236651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:30:50.241981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:50.243079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:50.244296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 613/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:50.249170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:50.347115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56bb2-548c-4619-9340-1aadff4931a3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:50.371319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:50.381577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:50.382173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:50.382450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:50.382720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:50.383378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:50.383680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:50.385069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:50.392950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 610/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:30:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:50.398690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:50.444193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:50.501288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec276a-d56e-4b10-9468-3d384cd19e02 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:50.535542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:50.536678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:50.537053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:50.537325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:50.547175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:50.550619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:50.564188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:50.565294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:50.566565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:50.566565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:50.566565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:50.566565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 614/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:50.571215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:50.670348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c5ac7-e9f9-47d1-9610-88f9fc3e42b6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:50.702247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:50.702852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:50.703123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:50.703398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:50.704021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:50.704346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:50.708845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:50.717551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 611/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:50.723214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:50.822248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:50.828762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ced939-431d-4107-8db8-1499c9173261 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:50.862348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:50.863142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:50.863419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:50.863689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:50.870866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:50.872331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:30:50.883641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:50.887998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:50.889536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 615/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:50.890974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:50.938885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:50.991744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b99bc-d104-4b57-ad67-6a6480f51434 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:51.025274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:51.025870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:51.026142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:51.026405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:51.027115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:51.027414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:51.028872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:51.045704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 612/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Sep 7 14:30:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:51.051615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:51.164563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca539-3d08-430a-b35a-7100a76882ce None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:51.197008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:51.197572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:51.197850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:51.198123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:51.209339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:51.210775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:51.224963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:51.226089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:51.227016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:51.227016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:51.227016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:51.227016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 616/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:51.231895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:51.330125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee0af-1a95-4549-8a57-d7a2e5963b9e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:51.332443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:51.363229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:51.363859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:51.364138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:51.365122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:51.366099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:51.366394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:51.367756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:51.378702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 613/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:51.384658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:51.394199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:51.483832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0b443-9576-4476-b4ab-2b28e78e4a33 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:51.519680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:51.520335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:51.521131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:51.521400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:51.527803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:51.529277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:30:51.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:51.541916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:51.543055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 617/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:30:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:51.547910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:51.644639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8fe99-549f-4b29-bd67-e8c3b8db7779 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:51.677678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:51.678281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:51.678553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:51.678818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:51.679562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:51.679856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:51.681292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c0a32-753d-4eb3-8a6b-4bd056559a0f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:51.693056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 614/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:51.699082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:51.765584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:51.801196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d31d0-e8d7-4da8-8f14-dad230d77118 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:51.835525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcd935-2956-4242-ba5a-5dd256a3be45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:51.836261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcd935-2956-4242-ba5a-5dd256a3be45 None admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:51.836489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcd935-2956-4242-ba5a-5dd256a3be45 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:51.836751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcd935-2956-4242-ba5a-5dd256a3be45 None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/53d54b01ab3c4003b5f13e20c0849db1/roles/ff7d54f17c444984871223a3bd3d7c78` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:51.851016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcd935-2956-4242-ba5a-5dd256a3be45 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=53d54b01ab3c4003b5f13e20c0849db1, role_id=ff7d54f17c444984871223a3bd3d7c78)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:51.852497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcd935-2956-4242-ba5a-5dd256a3be45 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:51.944941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 618/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:30:51 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/53d54b01ab3c4003b5f13e20c0849db1/roles/ff7d54f17c444984871223a3bd3d7c78 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:30:54.056256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a264c5-a319-402c-a9cc-c02f6539b213 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:54.056803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a264c5-a319-402c-a9cc-c02f6539b213 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:54.056896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a264c5-a319-402c-a9cc-c02f6539b213 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:54.058226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 615/1233] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:30:54.063634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:54.064061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:54.065593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:54.082397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:30:54.084212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:30:54.153731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 619/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:30:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:30:54.424344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a264c5-a319-402c-a9cc-c02f6539b213 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:54.449516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:54.525807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a264c5-a319-402c-a9cc-c02f6539b213 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:54.559117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:54.559725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:54.559997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:54.560291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:54.570488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:30:54.575232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:54.576346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:54.577419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 616/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Sep 7 14:30:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:54.582479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:54.629116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:54.679809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bc3d8-573c-4900-9165-007bae236b11 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:54.712118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:54.712737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:54.713008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:54.713335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:54.714047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:54.714341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:54.715881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:54.724315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 620/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Sep 7 14:30:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:54.730715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:54.814955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:54.830532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8942a1e-a089-4e10-bbce-66829aaede8b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:54.863291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:54.863906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:54.864202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:54.865081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:54.874608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:54.876038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:54.890816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:54.891943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:54.897397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:54.897397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:54.897397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:54.897397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 617/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:54.898502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:54.999361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710dcca-0910-4dde-bc90-fb122dcb9b77 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:55.008595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:55.033656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:55.034255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:55.034534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:55.034801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:55.035433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:55.035732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:55.037192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:55.048533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 621/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:55.054626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:55.154751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bfdf9-70ee-43c1-a6b0-96ca414823b5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:55.187372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:55.187996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:55.188313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:55.188584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:55.191310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:55.196939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:55.199181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:30:55.209185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:55.215237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:55.216612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 618/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:55.217329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:55.315507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264158c-12a0-49e2-a3db-be6c387e5766 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:55.349312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:55.349999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:55.350275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:55.350538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:55.351260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:55.351547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:55.352995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:55.370054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 622/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Sep 7 14:30:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:55.375789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:55.483425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf503c5-acff-4e72-81d4-87e314dc9128 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:55.517127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:55.517734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:55.518012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:55.518280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:55.578008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:55.584527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:55.585955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:55.683489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 619/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:30:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:30:55.689532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:55.716434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:55.788865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff5742-0e61-486a-bc1c-2fb44dee9090 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:55.822813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:55.823476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:55.823755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:55.824025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:55.840137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:55.840137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:55.850698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:55.851809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:55.852900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:55.852900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:55.852900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:55.852900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 623/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:55.858049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:55.951548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:55.957413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e613de-4a97-402b-bc9e-d99516e6c8b8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:55.990737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:55.991343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:55.991619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:55.991887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:55.992598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:55.992895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:55.994262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:56.004996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 620/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:56.011335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:56.090442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:56.111208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6664-a195-46bf-b536-9c19c34e947d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:56.144325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:56.144942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:56.145217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:56.145482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:56.151876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:56.153341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:30:56.164186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:56.165368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:30:56.166454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 624/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:56.171355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:56.268680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bbd4d-d57a-4bd2-96ff-2fc3cea61c8d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:56.302485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:56.303119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:56.303405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:56.303676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:56.304393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:56.304701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:56.306126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc18fa-c384-470d-9942-8b4fdab7c45b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:56.318364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 621/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:56.324558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:56.424847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd8429-b5fa-4000-bf72-e365a95fde52 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:56.458544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:56.459144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:56.459418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:56.459688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:56.460433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] RBAC: Adding query filter params (role.id=ff7d54f17c444984871223a3bd3d7c78, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=53d54b01ab3c4003b5f13e20c0849db1) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:56.460737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:56.462167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d717924-0670-4f00-ab2a-c93651d8cce5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:56.467844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 625/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:30:56 2022] GET /identity/v3/role_assignments?user.id=53d54b01ab3c4003b5f13e20c0849db1&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=ff7d54f17c444984871223a3bd3d7c78 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:58.693714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b9e6-6c7c-481d-b439-4fe37de624cb None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:58.694268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b9e6-6c7c-481d-b439-4fe37de624cb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:58.694268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b9e6-6c7c-481d-b439-4fe37de624cb None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:58.695667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 622/1247] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:30:58.700961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:58.701444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:58.701535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:58.707758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:58.719776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:30:58.720972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:30:58.785622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 626/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:30:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:30:59.062534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b9e6-6c7c-481d-b439-4fe37de624cb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:59.081104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:59.161429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b9e6-6c7c-481d-b439-4fe37de624cb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:59.195576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:59.196224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:59.196504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:59.196773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:59.206773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:30:59.211570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:59.215905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:59.217001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 623/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:30:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:59.218954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:59.319310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fce1e-5976-434e-bd0e-8f390f1213a7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:59.348665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:59.353471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:59.354067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:59.354347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:59.354677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:59.355336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:59.355639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:59.357029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:59.364943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 627/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:30:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:59.370797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:59.446536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:59.468676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a88ac-4653-4cfd-9a69-a9cae9db3ff2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:59.500437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c658e-e205-463e-a071-84807a466597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:59.500992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c658e-e205-463e-a071-84807a466597 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:59.501264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c658e-e205-463e-a071-84807a466597 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:59.501531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c658e-e205-463e-a071-84807a466597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:59.512698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c658e-e205-463e-a071-84807a466597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:59.514126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c658e-e205-463e-a071-84807a466597 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d76c658e-e205-463e-a071-84807a466597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:30:59.528523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:59.529672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:30:59.530740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:30:59.530740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:30:59.530740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:59.530740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 624/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:59.535537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:59.635014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b77d-53b6-44d6-837d-80af17a0eb79 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:59.672411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:59.673010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:59.673286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:59.673555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:59.674219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:59.674502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:59.675860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:59.687747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 628/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:30:59.694070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c658e-e205-463e-a071-84807a466597 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:59.793309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c658e-e205-463e-a071-84807a466597 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:59.820054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:59.827804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:59.828582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:59.828858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:59.829127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:59.836507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:59.837940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:30:59.848924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:30:59.849918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:30:59.850972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 625/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:30:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:30:59.856163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:30:59.911747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:30:59.955348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f49724-805b-4270-af54-9cf2e63664e0 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:30:59.988519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:30:59.989120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:30:59.991794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:30:59.992139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:30:59.992852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:30:59.993154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:30:59.994606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:00.013020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 629/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Sep 7 14:30:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:00.019455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:00.119118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff00b3-88ad-4456-a7ec-ca7e5f14302c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:00.152937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:00.153529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:00.153807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:00.154078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:00.208799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:00.220953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:00.222395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:00.318661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 626/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:31:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:31:00.324683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:00.423071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24ca8-a9aa-4836-9345-4a2d5cdcf628 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:00.437010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:00.457116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:00.457741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:00.458531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:00.458804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:00.468550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:00.470029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:31:00.484690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:00.485919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:31:00.487083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:31:00.487083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:00.487083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:00.487083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 630/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:00.491996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:00.578566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:00.590920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cfcc4-812e-416d-bf4c-00306ab32b2e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:00.624196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:00.624796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:00.625072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:00.625342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:00.625974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:00.626276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:00.627631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:00.638538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 627/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:00.644733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:00.743589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c054-06c3-4813-a722-36d47a4f089e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:00.776954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:00.777571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:00.777849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:00.778116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:00.784572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:00.786009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:31:00.797538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:00.798574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:00.799494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 631/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:00.804780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:00.902861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32f392-eb38-4a9f-ab78-31f7892eeff5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:00.935255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:00.935852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:00.936128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:00.936436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:00.937114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:00.937481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:00.938858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7a780-4be0-4e86-93d2-1d2d14c2c353 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:00.951587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 628/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:00.957803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:00.985297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:01.059471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a46e-c756-4cd6-abdf-0cd77b3cc25d None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:01.093205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:01.093803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:01.094080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:01.094410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:01.095129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=53d54b01ab3c4003b5f13e20c0849db1) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:01.095432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:01.096885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773e7d4-05b3-453a-b67d-7fbba3e23708 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:01.102161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 632/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:31:00 2022] GET /identity/v3/role_assignments?user.id=53d54b01ab3c4003b5f13e20c0849db1&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=c63cf2d08b334ece887cbd2702835968 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:03.217018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2652-30d2-404a-805d-11b21cdafe38 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:03.217639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2652-30d2-404a-805d-11b21cdafe38 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:03.217639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2652-30d2-404a-805d-11b21cdafe38 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:03.218896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 629/1261] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:31:03.224595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:03.224996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:03.225133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:03.243481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:31:03.244687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:31:03.308946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 633/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:31:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:31:03.577115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e2652-30d2-404a-805d-11b21cdafe38 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:03.588419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:03.676926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e2652-30d2-404a-805d-11b21cdafe38 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:03.710006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:03.710661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:03.710937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:03.711205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:03.720850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:31:03.726266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:03.727230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:03.728077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 630/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:03.733743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:03.751830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:03.832411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba967dc-ce6d-485c-9877-1418f0382679 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:03.865034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:03.865625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:03.865904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:03.866171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:03.866817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:03.867110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:03.868503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:03.876388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 634/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:31:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:03.882628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:03.949317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:03.981709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481c1aa-938f-4976-8e58-d0031f955e26 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.014963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.015596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.015866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.016130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.026964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:04.028412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:31:04.042349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:04.048620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:31:04.049755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:31:04.049755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:04.049755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:04.049755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 631/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:04.050023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:04.133127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:04.151958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed9349-bdeb-4a49-9193-d4af16ba82be None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.185624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.186221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.186501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.186773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.187412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:04.187714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:04.189105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:04.200906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 635/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:04.206723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:04.304566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732de69-0937-418b-b04e-0eca48361ef6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.324395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:04.339372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e26778-d444-405e-82ad-d986936945ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.340052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e26778-d444-405e-82ad-d986936945ce None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.340361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e26778-d444-405e-82ad-d986936945ce None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.340646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e26778-d444-405e-82ad-d986936945ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.348109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e26778-d444-405e-82ad-d986936945ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:04.349584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e26778-d444-405e-82ad-d986936945ce None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-90e26778-d444-405e-82ad-d986936945ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:31:04.360552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:04.361672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:04.362802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 632/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:04.368197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:04.468094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325174d-57bf-4c4b-a789-461ded90082c None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.500603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.501187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.501459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.501723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.502428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:04.502727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:04.504136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:04.513740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:04.521817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 636/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Sep 7 14:31:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:04.527872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e26778-d444-405e-82ad-d986936945ce None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:04.626620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e26778-d444-405e-82ad-d986936945ce None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.660622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.661280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.661556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.661828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.672557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:04.678239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:31:04.688136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:04.689249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:31:04.690393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:31:04.690393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:04.690393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:04.690393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 633/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:04.695873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:04.800340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed34e8d-9dfe-4921-89c1-312e0b83b09b None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.834672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.835267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.835565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.835832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.836658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:04.836814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:04.838243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:04.849086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 637/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:04.855082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:04.887429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:04.956365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0483ff-cac2-40de-9899-e678cbd49714 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:04.990154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:04.990941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:04.991832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:04.992087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:04.998659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:05.000077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:31:05.014508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:05.015629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:05.016782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 634/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:05.022345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:05.080481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:05.121300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36f295-82d5-4071-bfe2-f194f7e968c5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:05.154523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:05.155119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:05.155397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:05.155668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:05.156370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:05.156671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:05.158262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b099e9-ccfb-43ab-88e4-7a45685823f9 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:05.174306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 638/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:31:05.180642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:05.271536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:05.279986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd721-9be0-40a4-9d2c-e143c96644cc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:05.313075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f0315-76dd-461a-8b16-f6997818e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:05.313728 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f0315-76dd-461a-8b16-f6997818e6da None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:05.314025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f0315-76dd-461a-8b16-f6997818e6da None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:05.314403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f0315-76dd-461a-8b16-f6997818e6da None admin] PATH_INFO: `/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/53d54b01ab3c4003b5f13e20c0849db1/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:05.316285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f0315-76dd-461a-8b16-f6997818e6da None admin] RBAC: Authorizing `identity:create_grant(project_id=a0f1bc97dae24914a603354fcc9a8d41, user_id=53d54b01ab3c4003b5f13e20c0849db1, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:05.318196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f0315-76dd-461a-8b16-f6997818e6da None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:05.407395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 635/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Sep 7 14:31:05 2022] PUT /identity/v3/projects/a0f1bc97dae24914a603354fcc9a8d41/users/53d54b01ab3c4003b5f13e20c0849db1/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:31:07.553430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92093b3f-2b2b-40e0-896f-d0b71b1e9ffe None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:07.553945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92093b3f-2b2b-40e0-896f-d0b71b1e9ffe None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:07.553945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92093b3f-2b2b-40e0-896f-d0b71b1e9ffe None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:07.555312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 639/1274] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:31:07.560599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:07.561110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:07.561214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:07.579045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:31:07.580285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:31:07.646921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 636/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:31:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:31:07.914078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92093b3f-2b2b-40e0-896f-d0b71b1e9ffe None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.017728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92093b3f-2b2b-40e0-896f-d0b71b1e9ffe None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:08.035294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:08.051967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:08.052615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:08.052893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:08.053162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:08.062904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 14:31:08.067690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:08.068916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:08.069937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 640/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Sep 7 14:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:08.074830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.156470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:08.174215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ee08-5704-4ade-9409-48d66d34c585 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:08.207721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:08.208349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:08.208624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:08.208945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:08.209595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:08.220078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:08.220078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:08.220078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 637/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Sep 7 14:31:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:08.225156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.326559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59712ddf-6289-406c-9c26-98442c69addd None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:08.360539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:08.361152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:08.361429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:08.361699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:08.371957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:08.373544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:31:08.387613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:08.388683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:31:08.389924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:31:08.389924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:08.389924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:08.389924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 641/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:08.394754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.505075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355313ee-02d5-4306-a777-559059e4a3ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:08.541792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:08.542412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:08.542688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:08.542956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:08.543834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:08.544131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:08.545534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:08.564440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 638/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:08.565680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.598544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:08.676008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd40ad-1320-47f8-808f-894ba8a3b411 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:08.712407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:08.713056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:08.713332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:08.713599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:08.721278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:08.722912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 14:31:08.734717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:08.735830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:08.736883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 642/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 14:31:08.742466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.746412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:08.845416 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d16d87-9890-4968-b438-95e5367d52db None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:08.880788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:08.881390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:08.881668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:08.881945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:08.882664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:08.883000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:08.884625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:08.901999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 639/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Sep 7 14:31:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:08.907962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:08.998474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:09.008339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52920df3-717b-4dff-b2fc-ca3f1f0b84c5 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:09.042208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:09.042806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:09.043083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:09.043355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:09.110990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:09.112409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:09.204751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:09.211252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 643/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Sep 7 14:31:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 14:31:09.217322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:09.278501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:09.315828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd601ab0-60e5-4137-a58a-e8303850ac87 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:09.349492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:09.350143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:09.350423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:09.350692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:09.360589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:09.362068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 14:31:09.376846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:09.377966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 14:31:09.379119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 14:31:09.379119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 14:31:09.379119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:09.379119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 640/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:09.383658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:09.481337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f6227-9f08-4f92-8f35-8242e7333cbf None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:09.519695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:09.520386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:09.520664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:09.520931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:09.521563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:09.521869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:09.523231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:09.534285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 644/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Sep 7 14:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:09.540104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:09.643365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ddc1-797b-4749-ba40-5eb27611a1a0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:09.660484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:09.680970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:09.681596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:09.681875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:09.682146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:09.688735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:09.691320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 14:31:09.701923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:09.703113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Sep 07 14:31:09.704495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 641/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Sep 7 14:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:09.709003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:09.759476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:09.808878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd7287-2afd-4707-b9af-a71f4e9fd6c4 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:09.841163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:09.841762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:09.842043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:09.842339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:09.843023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:09.843326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:09.844788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd865-59e8-45bd-bc9f-522b40c7f2a4 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:09.856914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 645/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Sep 7 14:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:09.862880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:09.960621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729777-970b-43b4-8a01-9e2048dc6e3e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:09.995237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:09.995835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:09.996105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:09.996399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:09.997107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] RBAC: Adding query filter params (role.id=c63cf2d08b334ece887cbd2702835968, scope.project.id=a0f1bc97dae24914a603354fcc9a8d41, user.id=53d54b01ab3c4003b5f13e20c0849db1) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:09.997516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:09.998951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c8cd0-7c20-4210-b210-03ce6a142217 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:10.004599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 642/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Sep 7 14:31:09 2022] GET /identity/v3/role_assignments?user.id=53d54b01ab3c4003b5f13e20c0849db1&scope.project.id=a0f1bc97dae24914a603354fcc9a8d41&role.id=c63cf2d08b334ece887cbd2702835968 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:12.203684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55082832-3b8a-48f0-b17d-c78bf9343875 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:12.204222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55082832-3b8a-48f0-b17d-c78bf9343875 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:12.204332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55082832-3b8a-48f0-b17d-c78bf9343875 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:12.205611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 646/1288] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:31:12.211070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:12.211523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:12.211623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:12.231331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:31:12.232506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:31:12.263295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:12.300087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 643/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:31:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:31:12.566786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55082832-3b8a-48f0-b17d-c78bf9343875 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:12.668537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55082832-3b8a-48f0-b17d-c78bf9343875 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:12.679626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:12.702942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:12.703562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:12.703839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:12.704107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:12.704715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:12.706082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:12.714999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:12.720379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:12.721544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 647/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:31:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:12.723775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:12.821914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158d98f-c30b-4f41-aba8-2272352868f8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:12.854785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61767d-6905-42d4-993e-929df809f235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:12.855376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61767d-6905-42d4-993e-929df809f235 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:12.855651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61767d-6905-42d4-993e-929df809f235 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:12.855920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61767d-6905-42d4-993e-929df809f235 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:12.856570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61767d-6905-42d4-993e-929df809f235 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:12.856868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61767d-6905-42d4-993e-929df809f235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:12.858219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61767d-6905-42d4-993e-929df809f235 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:12.862892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 644/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Sep 7 14:31:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:12.869362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:12.966356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a952a-c896-41de-8dfa-8978594adf33 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:13.000538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:13.001137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:13.001415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:13.001685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] PATH_INFO: `/v3/services` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:13.002303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:13.002604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:13.004029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51c83-51ca-488b-9f35-3b82b8a31fc5 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:13.008370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 648/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Sep 7 14:31:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:15.165411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be17d0-6ca1-4beb-b2c0-346e7367a8d7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:15.165963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be17d0-6ca1-4beb-b2c0-346e7367a8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:15.165963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be17d0-6ca1-4beb-b2c0-346e7367a8d7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:15.167409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 645/1293] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:31:15.172883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced75af-9f46-42af-8e2f-4d18b7c161e6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:15.173260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced75af-9f46-42af-8e2f-4d18b7c161e6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:15.173425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced75af-9f46-42af-8e2f-4d18b7c161e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:15.191546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0ced75af-9f46-42af-8e2f-4d18b7c161e6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:31:15.193297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0ced75af-9f46-42af-8e2f-4d18b7c161e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:31:15.222918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:15.261937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 649/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:31:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:31:15.538308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be17d0-6ca1-4beb-b2c0-346e7367a8d7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:15.570009 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:15.639447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be17d0-6ca1-4beb-b2c0-346e7367a8d7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:15.672239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7025a1-0f55-47dd-9850-fa6706beb6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:15.672825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7025a1-0f55-47dd-9850-fa6706beb6b7 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:15.673098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7025a1-0f55-47dd-9850-fa6706beb6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:15.673362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7025a1-0f55-47dd-9850-fa6706beb6b7 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:15.674000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7025a1-0f55-47dd-9850-fa6706beb6b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:15.675341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7025a1-0f55-47dd-9850-fa6706beb6b7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:15.747804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 646/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Sep 7 14:31:15 2022] POST /identity/v3/services => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:31:17.924785 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88a1d7-7608-4af2-8830-6f3380e08f08 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:17.925363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88a1d7-7608-4af2-8830-6f3380e08f08 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:17.925363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88a1d7-7608-4af2-8830-6f3380e08f08 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:17.927228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 650/1296] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:31:17.932577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:17.933137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:17.933137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:17.953925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:31:17.955144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:31:18.028634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 647/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:31:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 14:31:18.304238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88a1d7-7608-4af2-8830-6f3380e08f08 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:18.411259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88a1d7-7608-4af2-8830-6f3380e08f08 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:18.443646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:18.444290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:18.444567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:18.449057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:18.449057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:18.449057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:18.455586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:18.456792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:18.458209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 651/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:31:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:18.463466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:18.482032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:18.560722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436b35-916d-416e-9ee7-177a7e95aeb2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:18.593591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:18.594193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:18.594472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:18.594745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:18.595370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:18.595668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:18.597055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:18.603251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 648/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Sep 7 14:31:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:18.609670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:18.642996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:18.708346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff6f83-40e6-44a4-8b4c-8ea1a6d5453e None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:18.742240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:18.742831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:18.743107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:18.743376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:18.744065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] RBAC: Adding query filter params (interface=public, service_id=fcf09b74a69049efb0281e92cb360ced, region_id=RegionOne) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:18.744398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:18.745770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792d4a6-05e2-41e9-b076-6e66272c0d91 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:18.750620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 652/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Sep 7 14:31:18 2022] GET /identity/v3/endpoints?service_id=fcf09b74a69049efb0281e92cb360ced&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:18.756866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:18.836594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:18.854400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1076-1884-488b-95e4-b85f78e8ba7c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:18.887335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0e6fd-8b19-420f-981c-8cf9e640b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:18.887934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0e6fd-8b19-420f-981c-8cf9e640b89c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:18.888240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0e6fd-8b19-420f-981c-8cf9e640b89c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:18.888512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0e6fd-8b19-420f-981c-8cf9e640b89c None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:18.889111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0e6fd-8b19-420f-981c-8cf9e640b89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:18.890733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0e6fd-8b19-420f-981c-8cf9e640b89c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:18.898193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 649/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Sep 7 14:31:18 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:31:21.028953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26b28d-6916-4844-ab42-582c58a44ead None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:21.029472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26b28d-6916-4844-ab42-582c58a44ead None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:21.029472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26b28d-6916-4844-ab42-582c58a44ead None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:21.030822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 653/1302] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:31:21.036593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:21.037039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:21.037171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:21.055919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:31:21.057139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:31:21.121262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 650/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:31:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 14:31:21.389952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26b28d-6916-4844-ab42-582c58a44ead None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:21.482644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:21.489752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26b28d-6916-4844-ab42-582c58a44ead None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:21.522733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:21.523348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:21.523711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:21.524208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:21.524977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:21.526719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 14:31:21.535310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 14:31:21.536515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application with Lock( Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self._enter() Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application return self.creator() Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application created_value = creator( Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: ERROR keystone.server.flask.application Sep 07 14:31:21.537363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 654/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Sep 7 14:31:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 14:31:21.542459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:21.618106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:21.640283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de03d06-653f-42d2-9af3-8e2a6a4418fd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:21.674721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:21.675322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:21.675597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:21.675862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:21.676541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] RBAC: Adding query filter params (name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:31:21.676840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:21.678203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daef688-9452-405d-8ef9-62a1ee7662db None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:21.683226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 651/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Sep 7 14:31:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:31:21.689092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:31:21.790900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091212e-303a-40e8-b1e1-c6a4b58699b6 None admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:31:21.824329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb205ff-079d-4571-af72-2e453c4959a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:31:21.824925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb205ff-079d-4571-af72-2e453c4959a0 None admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:31:21.825204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb205ff-079d-4571-af72-2e453c4959a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:31:21.825474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb205ff-079d-4571-af72-2e453c4959a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:31:21.826144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb205ff-079d-4571-af72-2e453c4959a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:31:21.827517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb205ff-079d-4571-af72-2e453c4959a0 None admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:31:21.931044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 655/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:31:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:31:31.680204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=30 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:31:31.926120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:21.808414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8611-1bd5-4a68-8ef4-744745cc5491 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:21.808974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8611-1bd5-4a68-8ef4-744745cc5491 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:21.808974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8611-1bd5-4a68-8ef4-744745cc5491 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:21.815936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a089a7d-2c5f-40b2-8332-94f6935e3825 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:21.816596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a089a7d-2c5f-40b2-8332-94f6935e3825 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:21.816596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a089a7d-2c5f-40b2-8332-94f6935e3825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:21.819486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 652/1307] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:21.899513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1a089a7d-2c5f-40b2-8332-94f6935e3825 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:21.900774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1a089a7d-2c5f-40b2-8332-94f6935e3825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:22.018351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 656/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:37:22.322255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d0093-cb2a-4ba7-80b8-feda654e84bf None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:22.322541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d0093-cb2a-4ba7-80b8-feda654e84bf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:22.322813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d0093-cb2a-4ba7-80b8-feda654e84bf None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:22.324328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 653/1309] 10.222.0.57 () {54 vars in 965 bytes} [Wed Sep 7 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:22.330282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c2d0-0d73-45b5-8e04-ae832c19cd24 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:22.330458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c2d0-0d73-45b5-8e04-ae832c19cd24 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:22.330734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c2d0-0d73-45b5-8e04-ae832c19cd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:22.412439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7158c2d0-0d73-45b5-8e04-ae832c19cd24 None None] MFA Rules not processed for user `b50c8a35e28e44bba3a4b4e56057a3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:22.413653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7158c2d0-0d73-45b5-8e04-ae832c19cd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:22.435713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:22.493580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 657/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Wed Sep 7 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:22.502603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d0093-cb2a-4ba7-80b8-feda654e84bf None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:22.504492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fd0d0093-cb2a-4ba7-80b8-feda654e84bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:22.632296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d0093-cb2a-4ba7-80b8-feda654e84bf None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:22.719056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:22.719682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:22.719959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:22.720281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:22.721987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:22.784534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:22.842345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:22.843920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f35d1-160b-48e7-acae-f048252a9a19 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:22.863006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 654/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:25.305564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f05c34-6910-44a9-844e-ef994b69308a None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:25.306113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f05c34-6910-44a9-844e-ef994b69308a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:25.306113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f05c34-6910-44a9-844e-ef994b69308a None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:25.307457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 658/1312] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:25.316234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3a97-ab05-441d-80e7-7b73a54ac5f9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:25.317310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3a97-ab05-441d-80e7-7b73a54ac5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:25.317594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3a97-ab05-441d-80e7-7b73a54ac5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:25.401683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-87cf3a97-ab05-441d-80e7-7b73a54ac5f9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:25.402918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-87cf3a97-ab05-441d-80e7-7b73a54ac5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:25.501277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 655/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:25.780463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f05c34-6910-44a9-844e-ef994b69308a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:25.781984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-08f05c34-6910-44a9-844e-ef994b69308a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:25.890486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f05c34-6910-44a9-844e-ef994b69308a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:25.961637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:25.963981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:25.964622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:25.964903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:25.965175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:25.966843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:26.074980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:26.077121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e328330-2a2f-472e-803d-bbeb8eca9cb8 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:26.095036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 659/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:28.778549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae4141-0d3d-40ce-a1da-c1efbaabd299 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:28.779586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae4141-0d3d-40ce-a1da-c1efbaabd299 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:28.779739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae4141-0d3d-40ce-a1da-c1efbaabd299 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:28.781395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 656/1315] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:28.786724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6db9-7e4d-4b23-a1dd-37496561d0c8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:28.787260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6db9-7e4d-4b23-a1dd-37496561d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:28.787260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6db9-7e4d-4b23-a1dd-37496561d0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:28.853711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-caaf6db9-7e4d-4b23-a1dd-37496561d0c8 None None] MFA Rules not processed for user `c95f9bc4dec644fdb5e6830927457a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:28.854857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-caaf6db9-7e4d-4b23-a1dd-37496561d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:28.907295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:28.951370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 660/1316] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:29.238373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae4141-0d3d-40ce-a1da-c1efbaabd299 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:29.239927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-59ae4141-0d3d-40ce-a1da-c1efbaabd299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:29.249641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:29.343674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae4141-0d3d-40ce-a1da-c1efbaabd299 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:29.414079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:29.414673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:29.414945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:29.415209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:29.416833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:29.477127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:29.524184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:29.525560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab0839-aa7c-494e-8159-b7a96358a3e1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:29.543838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 657/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:32.504545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2e8ba-dc03-43ca-885b-eccbe41e00ca None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:32.505106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2e8ba-dc03-43ca-885b-eccbe41e00ca None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:32.505106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2e8ba-dc03-43ca-885b-eccbe41e00ca None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:32.506445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 661/1318] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:32.511791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e453d-9fe7-446c-8ea2-29237f8ecf59 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:32.512219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e453d-9fe7-446c-8ea2-29237f8ecf59 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:32.512377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e453d-9fe7-446c-8ea2-29237f8ecf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:32.578289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-366e453d-9fe7-446c-8ea2-29237f8ecf59 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:32.579504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-366e453d-9fe7-446c-8ea2-29237f8ecf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:32.676980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 658/1319] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:32.962685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2e8ba-dc03-43ca-885b-eccbe41e00ca None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:32.964306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c3c2e8ba-dc03-43ca-885b-eccbe41e00ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:33.068983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2e8ba-dc03-43ca-885b-eccbe41e00ca None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:33.141057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:33.141646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:33.141920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:33.142187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:33.143781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:33.151118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:33.249566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:33.250943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb32b5b-9803-43ca-a75f-a63e9f87d357 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:33.269139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 662/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:36.379613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb86261-36f7-4763-805d-b5266fee5548 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:36.380193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb86261-36f7-4763-805d-b5266fee5548 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:36.380292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb86261-36f7-4763-805d-b5266fee5548 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:36.381629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 659/1321] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:36.387099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780651a3-56dd-4cb6-92b7-886070d7a7cb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:36.387547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780651a3-56dd-4cb6-92b7-886070d7a7cb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:36.387666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780651a3-56dd-4cb6-92b7-886070d7a7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:36.453701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-780651a3-56dd-4cb6-92b7-886070d7a7cb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:36.454902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-780651a3-56dd-4cb6-92b7-886070d7a7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:36.516462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:36.559329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 663/1322] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:36.840466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb86261-36f7-4763-805d-b5266fee5548 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:36.842030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fbb86261-36f7-4763-805d-b5266fee5548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:36.862003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:36.949660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb86261-36f7-4763-805d-b5266fee5548 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:37.024773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:37.025386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:37.025663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:37.025932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:37.027651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:37.100669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:37.148432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:37.149886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae0457-6126-4d80-ac41-33f70ef722a6 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:37.167658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 660/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:39.389674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a0ed-2dcc-4308-9ecc-00e1d732a4b4 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:39.390467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a0ed-2dcc-4308-9ecc-00e1d732a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:39.390467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a0ed-2dcc-4308-9ecc-00e1d732a4b4 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:39.391866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 664/1324] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:39.397772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83970-8888-47fc-9b1f-fb29938aa0fa None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:39.398306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83970-8888-47fc-9b1f-fb29938aa0fa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:39.398306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83970-8888-47fc-9b1f-fb29938aa0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:39.466779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-61e83970-8888-47fc-9b1f-fb29938aa0fa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:39.467993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61e83970-8888-47fc-9b1f-fb29938aa0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:39.564621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 661/1325] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:39.848898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045a0ed-2dcc-4308-9ecc-00e1d732a4b4 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:39.850519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c045a0ed-2dcc-4308-9ecc-00e1d732a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:39.954349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045a0ed-2dcc-4308-9ecc-00e1d732a4b4 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:40.033461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:40.034064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:40.034335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:40.034593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:40.036191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:40.053347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:40.147304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:40.148741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4fd8e-d134-4b56-954c-1f3850e6f422 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:40.167349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 665/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:42.366895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96619727-415a-4412-8046-f0b0738b2cf0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:42.367435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96619727-415a-4412-8046-f0b0738b2cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:42.367435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96619727-415a-4412-8046-f0b0738b2cf0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:42.368804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 662/1327] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:42.374631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de450b6-84cb-4b9d-9f95-041738fe1b33 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:42.375035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de450b6-84cb-4b9d-9f95-041738fe1b33 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:42.375169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de450b6-84cb-4b9d-9f95-041738fe1b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:42.440339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5de450b6-84cb-4b9d-9f95-041738fe1b33 None None] MFA Rules not processed for user `c95f9bc4dec644fdb5e6830927457a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:42.441466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5de450b6-84cb-4b9d-9f95-041738fe1b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:42.505685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:42.537462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 666/1328] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:42.838126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe892c-60dc-4e01-a4ab-476ee1843822 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:42.838406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe892c-60dc-4e01-a4ab-476ee1843822 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:42.838685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe892c-60dc-4e01-a4ab-476ee1843822 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:42.840025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 663/1329] 10.222.0.57 () {54 vars in 965 bytes} [Wed Sep 7 14:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:42.846878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cde16-6897-43a3-a2a4-ea707caf43ff None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:42.847241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cde16-6897-43a3-a2a4-ea707caf43ff None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:42.847581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cde16-6897-43a3-a2a4-ea707caf43ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:42.927386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7a7cde16-6897-43a3-a2a4-ea707caf43ff None None] MFA Rules not processed for user `b50c8a35e28e44bba3a4b4e56057a3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:42.928652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7a7cde16-6897-43a3-a2a4-ea707caf43ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:43.011007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 667/1330] 10.222.0.57 () {58 vars in 1062 bytes} [Wed Sep 7 14:37:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:43.020633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe892c-60dc-4e01-a4ab-476ee1843822 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:43.022157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-12fe892c-60dc-4e01-a4ab-476ee1843822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:43.053668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:43.129463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe892c-60dc-4e01-a4ab-476ee1843822 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:43.202730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:43.203337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:43.203611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:43.203880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:43.205485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:43.281580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:43.313696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:43.315203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f0f89-606a-4aa8-88e0-eed136569f12 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:43.335104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 664/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:46.445544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c129c6-3abe-4d56-8457-100f163f017f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:46.446083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c129c6-3abe-4d56-8457-100f163f017f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:46.446200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c129c6-3abe-4d56-8457-100f163f017f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:46.447513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 668/1332] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:46.452898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1a63b-7985-4147-aa20-c5ad2e83368e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:46.453315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1a63b-7985-4147-aa20-c5ad2e83368e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:46.453446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1a63b-7985-4147-aa20-c5ad2e83368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:46.520927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-67f1a63b-7985-4147-aa20-c5ad2e83368e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:46.522147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-67f1a63b-7985-4147-aa20-c5ad2e83368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:46.616358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 665/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:46.899610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c129c6-3abe-4d56-8457-100f163f017f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:46.901205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f1c129c6-3abe-4d56-8457-100f163f017f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:46.969678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:47.002992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c129c6-3abe-4d56-8457-100f163f017f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:47.076953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:47.077559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:47.077841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:47.078113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:47.079749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:47.188454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:47.189824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84d45a-cde9-40dd-8ab1-2707a7ba8014 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:47.199642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:47.208284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 669/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:49.430460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67a193-0631-4d41-be33-ffa99e9a1b96 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:49.431066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67a193-0631-4d41-be33-ffa99e9a1b96 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:49.431066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67a193-0631-4d41-be33-ffa99e9a1b96 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:49.432381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 666/1335] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:49.438193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a665e2-8e1a-4ab9-a534-0bda4f8dc80f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:49.438679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a665e2-8e1a-4ab9-a534-0bda4f8dc80f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:49.438779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a665e2-8e1a-4ab9-a534-0bda4f8dc80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:49.503044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-75a665e2-8e1a-4ab9-a534-0bda4f8dc80f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:49.504267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-75a665e2-8e1a-4ab9-a534-0bda4f8dc80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:49.599131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 670/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:49.882734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67a193-0631-4d41-be33-ffa99e9a1b96 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:49.884302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef67a193-0631-4d41-be33-ffa99e9a1b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:49.925192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:49.987915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67a193-0631-4d41-be33-ffa99e9a1b96 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:50.061623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:50.062229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:50.062506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:50.062777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:50.064401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:50.146886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:50.172541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:50.173972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c487c-fb5b-445e-91a6-774ca735ef62 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:50.191783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 667/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:53.022903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fe36b-208b-4649-8b14-18aa04d339d6 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:53.035332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fe36b-208b-4649-8b14-18aa04d339d6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:53.035332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fe36b-208b-4649-8b14-18aa04d339d6 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:53.035332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 671/1338] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:53.035638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206df1c-232c-4549-a6cc-1af09360540b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:53.035638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206df1c-232c-4549-a6cc-1af09360540b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:53.035638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206df1c-232c-4549-a6cc-1af09360540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:53.098952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c206df1c-232c-4549-a6cc-1af09360540b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:53.100336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c206df1c-232c-4549-a6cc-1af09360540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:53.196946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 668/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:53.476602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fe36b-208b-4649-8b14-18aa04d339d6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:53.478154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c18fe36b-208b-4649-8b14-18aa04d339d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:53.556508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:53.583926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fe36b-208b-4649-8b14-18aa04d339d6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:53.656933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:53.657453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:53.657724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:53.657989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:53.659555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:53.768594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:53.770096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafd064-c7de-43c1-8391-bc4ee345e45f service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:53.787930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 672/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:37:58.111759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299ec8a-6892-4d14-9a9f-e15a5d50e764 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:58.112309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299ec8a-6892-4d14-9a9f-e15a5d50e764 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:58.112309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299ec8a-6892-4d14-9a9f-e15a5d50e764 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:58.113725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 669/1341] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:37:58.118922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb33f0-6385-4452-975c-cf18f9f9b024 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:58.119334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb33f0-6385-4452-975c-cf18f9f9b024 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:58.119474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb33f0-6385-4452-975c-cf18f9f9b024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:58.125751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:58.189115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-85fb33f0-6385-4452-975c-cf18f9f9b024 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:37:58.190371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-85fb33f0-6385-4452-975c-cf18f9f9b024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:58.286515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 673/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:37:58.573338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299ec8a-6892-4d14-9a9f-e15a5d50e764 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:37:58.574903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7299ec8a-6892-4d14-9a9f-e15a5d50e764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:58.623886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:58.679759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299ec8a-6892-4d14-9a9f-e15a5d50e764 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:37:58.754136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:37:58.754740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:37:58.755015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:37:58.755285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:37:58.756902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:37:58.850235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:37:58.865906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:37:58.867340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311ad4b-f37a-4c9e-b0fa-cc0242decf11 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:37:58.885222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 670/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:01.998716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2f06e-8394-4b81-904e-52ed1a30b1e2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:01.999269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2f06e-8394-4b81-904e-52ed1a30b1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:01.999375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2f06e-8394-4b81-904e-52ed1a30b1e2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:02.000864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 674/1344] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:02.006147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c29c2-fe68-4aa5-9369-b6d4897c4985 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:02.006550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c29c2-fe68-4aa5-9369-b6d4897c4985 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:02.006691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c29c2-fe68-4aa5-9369-b6d4897c4985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:02.078583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a67c29c2-fe68-4aa5-9369-b6d4897c4985 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:02.079778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a67c29c2-fe68-4aa5-9369-b6d4897c4985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:02.183368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 671/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:02.469801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2f06e-8394-4b81-904e-52ed1a30b1e2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:02.471338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bfb2f06e-8394-4b81-904e-52ed1a30b1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:02.560887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:02.578777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2f06e-8394-4b81-904e-52ed1a30b1e2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:02.655280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:02.655888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:02.656224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:02.661518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:02.661518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:02.788996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:02.790479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f530e4-7869-4e54-ad24-79a69f5d58a9 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:02.810287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 675/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:07.355232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61868334-2ba5-49ae-adec-432873e197bd None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:07.355232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61868334-2ba5-49ae-adec-432873e197bd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:07.355232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61868334-2ba5-49ae-adec-432873e197bd None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:07.357138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 672/1347] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:07.364976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e192880-2b47-415b-8c91-235eda3eaf29 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:07.365846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e192880-2b47-415b-8c91-235eda3eaf29 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:07.366227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e192880-2b47-415b-8c91-235eda3eaf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:07.385957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:07.445268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3e192880-2b47-415b-8c91-235eda3eaf29 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:07.446404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3e192880-2b47-415b-8c91-235eda3eaf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:07.553365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 676/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:07.838536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61868334-2ba5-49ae-adec-432873e197bd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:07.840074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61868334-2ba5-49ae-adec-432873e197bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:07.897481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:07.942510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61868334-2ba5-49ae-adec-432873e197bd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:08.016621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:08.017235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:08.017513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:08.017865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:08.019491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:08.124432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:08.128027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:08.129553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243300d-75a7-4724-90f4-c86da77be025 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:08.152664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 673/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:12.962293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79640f-3f67-4ffe-b7c6-fd17cddd603b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:12.962806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79640f-3f67-4ffe-b7c6-fd17cddd603b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:12.962885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79640f-3f67-4ffe-b7c6-fd17cddd603b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:12.966670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 677/1350] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:12.971994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c5c82-ee03-4d70-ba4b-db2874a2050d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:12.972478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c5c82-ee03-4d70-ba4b-db2874a2050d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:12.972563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c5c82-ee03-4d70-ba4b-db2874a2050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:13.066112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bd7c5c82-ee03-4d70-ba4b-db2874a2050d None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:13.067566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bd7c5c82-ee03-4d70-ba4b-db2874a2050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:13.193337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 674/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:38:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:17.552535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=59 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:17.557695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83106e0c-f1f5-4ba4-ae93-a96083efa8aa None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:17.557992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83106e0c-f1f5-4ba4-ae93-a96083efa8aa None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:17.558278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83106e0c-f1f5-4ba4-ae93-a96083efa8aa None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:17.560454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 678/1352] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:17.565428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c652e2-a629-4e84-b822-960688aa1136 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:17.565862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c652e2-a629-4e84-b822-960688aa1136 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:17.565983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c652e2-a629-4e84-b822-960688aa1136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:17.622676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:17.646162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-73c652e2-a629-4e84-b822-960688aa1136 None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:17.647395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-73c652e2-a629-4e84-b822-960688aa1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:17.760094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 675/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:38:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:19.157798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808303-df6d-4a02-bca2-7e9ef1721443 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:19.158313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808303-df6d-4a02-bca2-7e9ef1721443 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:19.158313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808303-df6d-4a02-bca2-7e9ef1721443 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:19.159818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 679/1354] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:38:19.164946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5097731-f0d9-43c1-98ef-b8abd491b578 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:19.165362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5097731-f0d9-43c1-98ef-b8abd491b578 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:19.165638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5097731-f0d9-43c1-98ef-b8abd491b578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:19.242865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d5097731-f0d9-43c1-98ef-b8abd491b578 None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:19.243986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d5097731-f0d9-43c1-98ef-b8abd491b578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:19.289176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:19.353861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 676/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:38:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:38:23.038726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313e8e9-6c6c-4dae-95e4-e82d9b92e4d2 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:23.039713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313e8e9-6c6c-4dae-95e4-e82d9b92e4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:23.040239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313e8e9-6c6c-4dae-95e4-e82d9b92e4d2 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:23.053024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 680/1356] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:23.053169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4afcb-45d7-4d0d-ac84-c915ab571785 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:23.053169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4afcb-45d7-4d0d-ac84-c915ab571785 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:23.053169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4afcb-45d7-4d0d-ac84-c915ab571785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:23.123440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-82c4afcb-45d7-4d0d-ac84-c915ab571785 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:23.124600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-82c4afcb-45d7-4d0d-ac84-c915ab571785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:23.226765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 677/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:38:23.530753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994e7c-88dd-4640-ba73-ab6ba775101b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:23.531104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994e7c-88dd-4640-ba73-ab6ba775101b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:23.531610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994e7c-88dd-4640-ba73-ab6ba775101b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:23.533201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 681/1358] 10.222.0.57 () {54 vars in 962 bytes} [Wed Sep 7 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:23.540831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5691a-dcfa-4d43-8d99-b0a17c1185b5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:23.541393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5691a-dcfa-4d43-8d99-b0a17c1185b5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:23.542002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5691a-dcfa-4d43-8d99-b0a17c1185b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:23.562055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:23.624473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b0b5691a-dcfa-4d43-8d99-b0a17c1185b5 None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:23.625788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b0b5691a-dcfa-4d43-8d99-b0a17c1185b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:23.774333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 678/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Sep 7 14:38:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:23.783391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994e7c-88dd-4640-ba73-ab6ba775101b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:23.785072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d9994e7c-88dd-4640-ba73-ab6ba775101b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:23.947604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994e7c-88dd-4640-ba73-ab6ba775101b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:24.037000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:24.080205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:24.080865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:24.081542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:24.081861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:24.083781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:24.180621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:24.182264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb5d8d-db4a-4aab-8f73-d7b7b7a55455 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:24.202124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 682/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:38:26.572222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e66bb-1e7d-497c-994f-12c36dcfb4b3 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:26.572743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e66bb-1e7d-497c-994f-12c36dcfb4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:26.572743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e66bb-1e7d-497c-994f-12c36dcfb4b3 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:26.578653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 679/1361] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:26.587973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef0179-4e78-4af3-9208-3ba69c05009d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:26.588579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef0179-4e78-4af3-9208-3ba69c05009d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:26.588579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef0179-4e78-4af3-9208-3ba69c05009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:26.679345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-40ef0179-4e78-4af3-9208-3ba69c05009d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:26.680584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-40ef0179-4e78-4af3-9208-3ba69c05009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:26.723421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:26.783790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 683/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:27.075577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e66bb-1e7d-497c-994f-12c36dcfb4b3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:27.077312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-956e66bb-1e7d-497c-994f-12c36dcfb4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:27.147106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:27.253479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e66bb-1e7d-497c-994f-12c36dcfb4b3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:27.392327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:27.392947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:27.393230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:27.393498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:27.395115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:27.418189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:27.487330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:27.488995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d0508-b2ba-4203-bb74-ebfab1e9cbc4 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:27.507310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 680/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:30.133714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e7e84-e7c2-405f-a697-4480007ee7bb None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:30.134780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e7e84-e7c2-405f-a697-4480007ee7bb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:30.135348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e7e84-e7c2-405f-a697-4480007ee7bb None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:30.138302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 684/1364] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:30.143336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc86a7f-8934-402e-8251-f5aaec1ed9e0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:30.143733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc86a7f-8934-402e-8251-f5aaec1ed9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:30.143915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc86a7f-8934-402e-8251-f5aaec1ed9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:30.213357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6dc86a7f-8934-402e-8251-f5aaec1ed9e0 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:30.214581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6dc86a7f-8934-402e-8251-f5aaec1ed9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:30.317137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:30.320191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 681/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:30.617768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d99003-bc67-4380-a13d-dff0b3d6bda8 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:30.618302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d99003-bc67-4380-a13d-dff0b3d6bda8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:30.618664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d99003-bc67-4380-a13d-dff0b3d6bda8 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:30.620291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 685/1366] 10.222.0.57 () {54 vars in 962 bytes} [Wed Sep 7 14:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:30.626270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e32e-3964-43ed-af16-95e7d6aa0d0d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:30.626622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e32e-3964-43ed-af16-95e7d6aa0d0d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:30.626964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8e32e-3964-43ed-af16-95e7d6aa0d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:30.704170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-56d8e32e-3964-43ed-af16-95e7d6aa0d0d None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:30.705382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-56d8e32e-3964-43ed-af16-95e7d6aa0d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:30.826407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 682/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Sep 7 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:30.834690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d99003-bc67-4380-a13d-dff0b3d6bda8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:30.836406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e9d99003-bc67-4380-a13d-dff0b3d6bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:31.003690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d99003-bc67-4380-a13d-dff0b3d6bda8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:31.025736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:31.129762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:31.130751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:31.131439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:31.131939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:31.133878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:31.173604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0844db8-7339-4e2f-8fdb-18bf7dde52c1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:31.174126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0844db8-7339-4e2f-8fdb-18bf7dde52c1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:31.174126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0844db8-7339-4e2f-8fdb-18bf7dde52c1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:31.176440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 683/1368] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:38:31.180428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f87ff-a681-4c7a-bcf8-452776af03be None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:31.180711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f87ff-a681-4c7a-bcf8-452776af03be None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:31.180985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f87ff-a681-4c7a-bcf8-452776af03be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:31.233073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:31.234616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e17a4-868a-48c0-b6b0-925fad39bc5a service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:31.240106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:31.247506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0e8f87ff-a681-4c7a-bcf8-452776af03be None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:31.248845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0e8f87ff-a681-4c7a-bcf8-452776af03be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:31.253963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 686/1369] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:31.356763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 684/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:38:31.698722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100b97-be8e-494a-ab96-34a08f7e7f9a None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:31.698722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100b97-be8e-494a-ab96-34a08f7e7f9a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:31.698722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100b97-be8e-494a-ab96-34a08f7e7f9a None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:31.701198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 687/1371] 10.222.0.57 () {54 vars in 964 bytes} [Wed Sep 7 14:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:31.705070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f851-d7d6-4184-bf5f-086faf590c06 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:31.705362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f851-d7d6-4184-bf5f-086faf590c06 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:31.705638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f851-d7d6-4184-bf5f-086faf590c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:31.787493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1113f851-d7d6-4184-bf5f-086faf590c06 None None] MFA Rules not processed for user `d83045f9a43b48d7bbcce339f7bfa5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:31.788838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1113f851-d7d6-4184-bf5f-086faf590c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:31.864369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:31.872814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 685/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Wed Sep 7 14:38:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:31.885228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100b97-be8e-494a-ab96-34a08f7e7f9a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:31.885228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d4100b97-be8e-494a-ab96-34a08f7e7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:31.911456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:32.001010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100b97-be8e-494a-ab96-34a08f7e7f9a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:32.086606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:32.087217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:32.087501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:32.087988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:32.089580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:32.179682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:32.218151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:32.219531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8534b2-ed30-4d2b-804a-56772bd702d3 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:32.242270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 688/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:33.618955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe995b-495f-460f-8012-062d43525a15 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:33.619467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe995b-495f-460f-8012-062d43525a15 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:33.619467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe995b-495f-460f-8012-062d43525a15 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:33.620930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 686/1374] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:33.626233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2e72-5f1c-4932-ab28-e5c0a6c65793 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:33.626640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2e72-5f1c-4932-ab28-e5c0a6c65793 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:33.626784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2e72-5f1c-4932-ab28-e5c0a6c65793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:33.693717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a8bf2e72-5f1c-4932-ab28-e5c0a6c65793 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:33.694972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a8bf2e72-5f1c-4932-ab28-e5c0a6c65793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:33.797134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 689/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:34.094783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe995b-495f-460f-8012-062d43525a15 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:34.096456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a4fe995b-495f-460f-8012-062d43525a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:34.254196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe995b-495f-460f-8012-062d43525a15 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:34.321899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:34.369824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:34.370432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:34.370707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:34.371616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:34.373236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:34.471119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:34.472770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f864ed-37ba-4e60-8f19-3c4ff0e28ba6 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:34.491962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 687/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:38:36.836849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be782a-f152-408f-8a64-9092911211a1 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:36.837451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be782a-f152-408f-8a64-9092911211a1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:36.837707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be782a-f152-408f-8a64-9092911211a1 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:36.839313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 690/1377] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:36.845059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0774f7-9c83-4f36-9073-d192b94d1bb5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:36.845478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0774f7-9c83-4f36-9073-d192b94d1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:36.845648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0774f7-9c83-4f36-9073-d192b94d1bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:36.914046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cb0774f7-9c83-4f36-9073-d192b94d1bb5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:36.915964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cb0774f7-9c83-4f36-9073-d192b94d1bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:36.949188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:37.024061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 688/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:37.315836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be782a-f152-408f-8a64-9092911211a1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:37.317512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-14be782a-f152-408f-8a64-9092911211a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:37.358920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:37.474293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be782a-f152-408f-8a64-9092911211a1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:37.598691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:37.604261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:37.604971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:37.605361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:37.605699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:37.607499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:37.703750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:37.705451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de17f5-f6c5-479e-8ab4-8540d09ccac6 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:37.724935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 691/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:40.041257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a429-a139-4244-9c75-df5d77a69803 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:40.041780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a429-a139-4244-9c75-df5d77a69803 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:40.041780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a429-a139-4244-9c75-df5d77a69803 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:40.045668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 689/1380] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:40.048925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54391e8-126b-4308-b2f3-d6529d7d1954 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:40.049376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54391e8-126b-4308-b2f3-d6529d7d1954 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:40.049470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54391e8-126b-4308-b2f3-d6529d7d1954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:40.120406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f54391e8-126b-4308-b2f3-d6529d7d1954 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:40.121658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f54391e8-126b-4308-b2f3-d6529d7d1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:40.195782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:40.236250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 692/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:40.529007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a429-a139-4244-9c75-df5d77a69803 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:40.531099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2e03a429-a139-4244-9c75-df5d77a69803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:40.703561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a429-a139-4244-9c75-df5d77a69803 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:40.717713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:40.827217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:40.827830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:40.828106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:40.828405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:40.830004 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:40.935454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:40.937750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53caa3b6-a04e-45a0-bf02-bf33275f4045 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:40.960286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 690/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:38:43.428404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3e4b-d0a1-4bad-ac05-cd3b23f16bf8 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:43.428930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3e4b-d0a1-4bad-ac05-cd3b23f16bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:43.429056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3e4b-d0a1-4bad-ac05-cd3b23f16bf8 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:43.430508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 693/1383] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:43.436532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486e6f-47b7-4f61-934b-5933189df4e8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:43.437171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486e6f-47b7-4f61-934b-5933189df4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:43.437267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f486e6f-47b7-4f61-934b-5933189df4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:43.462870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:43.514873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7f486e6f-47b7-4f61-934b-5933189df4e8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:43.516398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7f486e6f-47b7-4f61-934b-5933189df4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:43.616562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 691/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:43.910285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3e4b-d0a1-4bad-ac05-cd3b23f16bf8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:43.911812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-150e3e4b-d0a1-4bad-ac05-cd3b23f16bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:44.084131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3e4b-d0a1-4bad-ac05-cd3b23f16bf8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:44.152309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:44.230800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:44.231718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:44.232074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:44.232541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:44.234663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:44.344578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:44.346883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb57b4-3dba-4491-a4a6-7dc1d33d77f3 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:44.370420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 694/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:46.847273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07ec5-0e36-4f6a-9e91-fd0c32ebfe28 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:46.847886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07ec5-0e36-4f6a-9e91-fd0c32ebfe28 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:46.847886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07ec5-0e36-4f6a-9e91-fd0c32ebfe28 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:46.849183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 692/1386] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:46.854555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9446cf-6df4-4bd6-b111-3f6323ba9aac None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:46.855009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9446cf-6df4-4bd6-b111-3f6323ba9aac None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:46.855192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9446cf-6df4-4bd6-b111-3f6323ba9aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:46.924180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1f9446cf-6df4-4bd6-b111-3f6323ba9aac None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:46.925847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1f9446cf-6df4-4bd6-b111-3f6323ba9aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:46.934516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:47.026779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 695/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:47.322563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e07ec5-0e36-4f6a-9e91-fd0c32ebfe28 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:47.324559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-65e07ec5-0e36-4f6a-9e91-fd0c32ebfe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:47.415596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:47.470704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e07ec5-0e36-4f6a-9e91-fd0c32ebfe28 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:47.584252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:47.584778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:47.585054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:47.585324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:47.586901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:47.645853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:47.675183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:47.676826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44e7d0-9c1d-405b-84ea-abd91cdd49a0 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:47.695749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 693/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:47.935028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3849b-b956-4b57-929b-1a7fd4911f43 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:47.935645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3849b-b956-4b57-929b-1a7fd4911f43 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:47.935645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3849b-b956-4b57-929b-1a7fd4911f43 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:47.936991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 696/1389] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:47.942688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15870e-d017-4089-aefe-0fd594bf6879 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:47.943192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15870e-d017-4089-aefe-0fd594bf6879 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:47.943283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15870e-d017-4089-aefe-0fd594bf6879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:48.023199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5c15870e-d017-4089-aefe-0fd594bf6879 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:48.024494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5c15870e-d017-4089-aefe-0fd594bf6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:48.125184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 694/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:48.428451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af67239-bfa5-4a7a-91f8-e4ff7de4e94a None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:48.428744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af67239-bfa5-4a7a-91f8-e4ff7de4e94a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:48.429023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af67239-bfa5-4a7a-91f8-e4ff7de4e94a None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:48.430360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 697/1391] 10.222.0.57 () {54 vars in 961 bytes} [Wed Sep 7 14:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:38:48.435716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dfaae-0fbb-4d5c-984d-9ab8761729e3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:48.436000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dfaae-0fbb-4d5c-984d-9ab8761729e3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:48.436413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dfaae-0fbb-4d5c-984d-9ab8761729e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:48.482782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:48.515415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ef2dfaae-0fbb-4d5c-984d-9ab8761729e3 None None] MFA Rules not processed for user `113ccc244a814d7580d1c9f6319edf04`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:48.516720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef2dfaae-0fbb-4d5c-984d-9ab8761729e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:48.599360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 695/1392] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Sep 7 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:48.608743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af67239-bfa5-4a7a-91f8-e4ff7de4e94a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:48.610328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6af67239-bfa5-4a7a-91f8-e4ff7de4e94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:48.726153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af67239-bfa5-4a7a-91f8-e4ff7de4e94a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:48.773959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:48.813766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:48.814379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:48.814658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:48.814928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:48.816549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:48.932666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:48.935063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae3546-9259-420e-a911-abae61e61f2c service glance] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:48.954215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 698/1393] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:38:49.848202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014378-2822-43b9-9a47-e9b095076f07 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:49.848718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014378-2822-43b9-9a47-e9b095076f07 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:49.848814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014378-2822-43b9-9a47-e9b095076f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:49.920006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0b014378-2822-43b9-9a47-e9b095076f07 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:49.921293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0b014378-2822-43b9-9a47-e9b095076f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:50.009856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:50.018260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 696/1394] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:50.063602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a179d8-d90f-458b-bb03-d415edec7a3f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:50.064080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a179d8-d90f-458b-bb03-d415edec7a3f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:50.064357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a179d8-d90f-458b-bb03-d415edec7a3f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:50.065543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 699/1395] 10.222.0.57 () {54 vars in 943 bytes} [Wed Sep 7 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:50.071861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:50.072174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:50.072457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:50.078028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b02860-1099-4954-b58a-190a0b37010f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:50.078316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b02860-1099-4954-b58a-190a0b37010f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:50.078594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b02860-1099-4954-b58a-190a0b37010f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:50.079904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 700/1396] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:50.085091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352608fe-755f-450f-b113-fcde162db228 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:50.085404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352608fe-755f-450f-b113-fcde162db228 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:50.085677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352608fe-755f-450f-b113-fcde162db228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:50.151462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] MFA Rules not processed for user `256b9d949c8e4734a6bce965a107c060`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:50.154197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:50.159782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-352608fe-755f-450f-b113-fcde162db228 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:50.160991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-352608fe-755f-450f-b113-fcde162db228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:50.179233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:50.266563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 697/1397] 10.222.0.57 () {58 vars in 1040 bytes} [Wed Sep 7 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:50.269176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 701/1398] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:38:50.276013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:50.285867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df67c1e-1189-43a4-95a7-a407ee74e59b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:50.335182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:50.391092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:50.391727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:50.392005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:50.392313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:50.393959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:50.475143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:50.476768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26234538-868f-4c23-b8fd-63ebe019d593 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:50.484675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 698/1399] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:38:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:38:50.561944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352608fe-755f-450f-b113-fcde162db228 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:50.563151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-352608fe-755f-450f-b113-fcde162db228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:50.680607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352608fe-755f-450f-b113-fcde162db228 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:50.719444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:50.797509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:50.798110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:50.798440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:50.798660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:50.800294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:50.903097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:50.903972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23527e9f-9d87-4096-919b-55332ca3d5b8 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:50.931690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 702/1400] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:54.248770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00487d32-7723-4ea5-b9c9-adb0d1766fae None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:54.249674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00487d32-7723-4ea5-b9c9-adb0d1766fae None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:54.250188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00487d32-7723-4ea5-b9c9-adb0d1766fae None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:54.252016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 699/1401] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:54.257091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e61d63-326f-479d-8d71-6d60b69806c6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:54.257585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e61d63-326f-479d-8d71-6d60b69806c6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:54.257669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e61d63-326f-479d-8d71-6d60b69806c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:54.308995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:54.330944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-77e61d63-326f-479d-8d71-6d60b69806c6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:54.332232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-77e61d63-326f-479d-8d71-6d60b69806c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:54.438379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 703/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:54.729086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00487d32-7723-4ea5-b9c9-adb0d1766fae None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:54.731314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-00487d32-7723-4ea5-b9c9-adb0d1766fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:54.826904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:54.909989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00487d32-7723-4ea5-b9c9-adb0d1766fae None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:55.043711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:55.044357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:55.044637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:55.044906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:55.046634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:55.097469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:55.153739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:55.155523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc9566-e97c-4be9-a3d1-697df29d3466 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:55.174433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 700/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:38:57.476260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c389af6-b11a-4cb5-89d5-e1383897b58d None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:57.476818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c389af6-b11a-4cb5-89d5-e1383897b58d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:57.476818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c389af6-b11a-4cb5-89d5-e1383897b58d None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:57.478197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 704/1404] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:38:57.483695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ca5f-f16d-43ea-b0dd-ad426b596443 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:57.484461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ca5f-f16d-43ea-b0dd-ad426b596443 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:57.484907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ca5f-f16d-43ea-b0dd-ad426b596443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:57.553993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0952ca5f-f16d-43ea-b0dd-ad426b596443 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:38:57.555240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0952ca5f-f16d-43ea-b0dd-ad426b596443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:57.667504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 701/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:38:57.968201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c389af6-b11a-4cb5-89d5-e1383897b58d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:38:57.969812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2c389af6-b11a-4cb5-89d5-e1383897b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:58.105896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:58.144950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c389af6-b11a-4cb5-89d5-e1383897b58d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:38:58.282077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:38:58.283051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:38:58.283686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:38:58.283957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:38:58.285683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:38:58.374544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:38:58.384997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:38:58.386891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b689d14-d25f-4992-8823-4b1b76fbd853 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:38:58.409975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 705/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:00.723015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97d87-8b1b-486a-a0fb-4fb05b794094 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:00.723579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97d87-8b1b-486a-a0fb-4fb05b794094 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:00.723579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97d87-8b1b-486a-a0fb-4fb05b794094 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:00.728545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 702/1407] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:39:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:00.730673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641cd5-e021-44e8-87af-da46717cd1e8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:00.731138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641cd5-e021-44e8-87af-da46717cd1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:00.731234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641cd5-e021-44e8-87af-da46717cd1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:00.819736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a7641cd5-e021-44e8-87af-da46717cd1e8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:00.821030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a7641cd5-e021-44e8-87af-da46717cd1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:00.986365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 706/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:39:01.283277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97d87-8b1b-486a-a0fb-4fb05b794094 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:01.285344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-36d97d87-8b1b-486a-a0fb-4fb05b794094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:01.315184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:01.472566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97d87-8b1b-486a-a0fb-4fb05b794094 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:01.613252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:01.613903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:01.614184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:01.614463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:01.616066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:01.738187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:01.740710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6de005-5dc9-47f6-bc5e-f592953c1029 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:01.759374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 703/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:01.787241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:10.977610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=74 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:12.049873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=54 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:13.936375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c23f8-1515-403f-8f7d-78347946dd0f None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:13.936968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c23f8-1515-403f-8f7d-78347946dd0f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:13.936968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c23f8-1515-403f-8f7d-78347946dd0f None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:13.938340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 707/1410] 10.222.0.57 () {54 vars in 918 bytes} [Wed Sep 7 14:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:13.943496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d56b0-81b8-43de-96d1-253b4223d1f1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:13.943930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d56b0-81b8-43de-96d1-253b4223d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:13.944026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d56b0-81b8-43de-96d1-253b4223d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:14.020401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7b0d56b0-81b8-43de-96d1-253b4223d1f1 None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:14.021605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7b0d56b0-81b8-43de-96d1-253b4223d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:14.124456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 704/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Sep 7 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:14.139580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4756-953b-449d-af72-a90a46f1e193 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:14.139890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4756-953b-449d-af72-a90a46f1e193 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:14.140455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4756-953b-449d-af72-a90a46f1e193 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:14.141845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 708/1412] 10.222.0.57 () {54 vars in 918 bytes} [Wed Sep 7 14:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:14.147181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9236-7395-4cd5-9ca8-61af4c55f66e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:14.147463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9236-7395-4cd5-9ca8-61af4c55f66e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:14.147758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9236-7395-4cd5-9ca8-61af4c55f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:14.218980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:14.221599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-066d9236-7395-4cd5-9ca8-61af4c55f66e None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:14.222792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-066d9236-7395-4cd5-9ca8-61af4c55f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:14.336544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 705/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Sep 7 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:39:16.921576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5052e0-c313-4157-b2db-cd0577325820 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:16.922149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5052e0-c313-4157-b2db-cd0577325820 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:16.922149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5052e0-c313-4157-b2db-cd0577325820 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:16.923529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:16.928886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee51142-6feb-4c93-8239-21112fa01e82 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:16.929293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee51142-6feb-4c93-8239-21112fa01e82 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:16.929434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee51142-6feb-4c93-8239-21112fa01e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:16.996204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3ee51142-6feb-4c93-8239-21112fa01e82 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:16.997431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3ee51142-6feb-4c93-8239-21112fa01e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:17.051538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:17.093319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:17.385087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5052e0-c313-4157-b2db-cd0577325820 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:17.386638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2a5052e0-c313-4157-b2db-cd0577325820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:17.532770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5052e0-c313-4157-b2db-cd0577325820 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:17.605787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:17.646639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:17.647180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:17.647457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:17.647726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:17.649346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:17.736376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:17.737974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746a273-cf10-4b64-ae0c-451e89cbc90e service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:17.756682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 710/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:18.910368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362f9a1-061f-43a0-9ee5-84ef1980a4be None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:18.910910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362f9a1-061f-43a0-9ee5-84ef1980a4be None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:18.910910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362f9a1-061f-43a0-9ee5-84ef1980a4be None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:18.912199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:18.919180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257372a9-ad5b-4853-8e15-056b44b05f19 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:18.919671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257372a9-ad5b-4853-8e15-056b44b05f19 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:18.919780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257372a9-ad5b-4853-8e15-056b44b05f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:18.996233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-257372a9-ad5b-4853-8e15-056b44b05f19 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:18.997453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-257372a9-ad5b-4853-8e15-056b44b05f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:19.023721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:19.093729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:19.243964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362f9a1-061f-43a0-9ee5-84ef1980a4be None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:19.245461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6362f9a1-061f-43a0-9ee5-84ef1980a4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:19.389191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362f9a1-061f-43a0-9ee5-84ef1980a4be None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:19.429404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:19.501861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:19.502399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:19.502673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:19.502939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:19.504573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:19.590868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:19.592545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33357ae-1d61-4cb2-9cea-a6cc32ab7ed8 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:19.609911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 708/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:20.765919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb067e-3ab0-4f2f-9c9c-82f027cb4b45 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:20.766534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb067e-3ab0-4f2f-9c9c-82f027cb4b45 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:20.766534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb067e-3ab0-4f2f-9c9c-82f027cb4b45 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:20.767823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:20.774299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec048a5-b66a-43d2-a331-097cc9ff51aa None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:20.774818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec048a5-b66a-43d2-a331-097cc9ff51aa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:20.774914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec048a5-b66a-43d2-a331-097cc9ff51aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:20.830907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:20.853658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9ec048a5-b66a-43d2-a331-097cc9ff51aa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:20.854893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9ec048a5-b66a-43d2-a331-097cc9ff51aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:20.953547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:21.112037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb067e-3ab0-4f2f-9c9c-82f027cb4b45 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:21.113702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-78cb067e-3ab0-4f2f-9c9c-82f027cb4b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:21.262462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb067e-3ab0-4f2f-9c9c-82f027cb4b45 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:21.274916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:21.385424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:21.385942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:21.386217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:21.386484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:21.388085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:21.474749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:21.476356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de0dfa-dfd2-4002-83bd-ca1e1e64b5e5 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:21.495671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 713/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:22.593046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d744f8-726c-436e-a356-9be6c13cdafb None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:22.593641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d744f8-726c-436e-a356-9be6c13cdafb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:22.593641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d744f8-726c-436e-a356-9be6c13cdafb None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:22.594934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:22.601171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490b43b-ef32-4dad-8d8d-3424ade4e584 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:22.601584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490b43b-ef32-4dad-8d8d-3424ade4e584 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:22.601729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490b43b-ef32-4dad-8d8d-3424ade4e584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:22.621942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:22.678618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2490b43b-ef32-4dad-8d8d-3424ade4e584 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:22.679842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2490b43b-ef32-4dad-8d8d-3424ade4e584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:22.776042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:22.926941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d744f8-726c-436e-a356-9be6c13cdafb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:22.928405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-14d744f8-726c-436e-a356-9be6c13cdafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:23.036176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:23.072538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d744f8-726c-436e-a356-9be6c13cdafb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:23.182400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:23.182957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:23.183231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:23.183497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:23.185084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:23.257638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:23.270859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:23.272456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14adb6-160c-408f-ae3b-5a71eb6a36f3 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:23.290769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 711/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:24.414038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec3c88-11d7-439d-a88b-94e0c4395f70 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:24.414723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec3c88-11d7-439d-a88b-94e0c4395f70 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:24.414723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec3c88-11d7-439d-a88b-94e0c4395f70 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:24.415940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:24.422790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c365f-32e4-4025-b52d-3907a81887d8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:24.423197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c365f-32e4-4025-b52d-3907a81887d8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:24.423345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c365f-32e4-4025-b52d-3907a81887d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:24.499808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d20c365f-32e4-4025-b52d-3907a81887d8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:24.501046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d20c365f-32e4-4025-b52d-3907a81887d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:24.597999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:24.749050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec3c88-11d7-439d-a88b-94e0c4395f70 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:24.750502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-06ec3c88-11d7-439d-a88b-94e0c4395f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:24.834886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:24.897137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec3c88-11d7-439d-a88b-94e0c4395f70 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:25.013183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:25.013716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:25.013992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:25.014263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:25.016165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:25.064850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:25.105876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:25.107492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d0b3-d7ea-459e-a5c5-b7eba91221e9 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:25.125157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 716/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:26.253242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bbfa7-282b-4989-89d9-293a19978e1f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:26.253785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bbfa7-282b-4989-89d9-293a19978e1f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:26.253785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bbfa7-282b-4989-89d9-293a19978e1f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:26.255141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:26.262048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba228c4-33d8-4a6a-af4b-89c6ad551d38 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:26.262514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba228c4-33d8-4a6a-af4b-89c6ad551d38 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:26.262623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba228c4-33d8-4a6a-af4b-89c6ad551d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:26.337157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4ba228c4-33d8-4a6a-af4b-89c6ad551d38 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:26.338402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4ba228c4-33d8-4a6a-af4b-89c6ad551d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:26.437893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:26.587344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bbfa7-282b-4989-89d9-293a19978e1f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:26.588925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6c4bbfa7-282b-4989-89d9-293a19978e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:26.636875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:26.736000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bbfa7-282b-4989-89d9-293a19978e1f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:26.846561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:26.847115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:26.847391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:26.847659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:26.849277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:26.856560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:26.938684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:26.940351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d45368-5e37-4d9b-8e64-c44d2b0a1c39 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:26.957948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 714/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:28.071203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6162-e446-41cd-b53e-2bcf41ed8068 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:28.071838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6162-e446-41cd-b53e-2bcf41ed8068 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:28.071838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6162-e446-41cd-b53e-2bcf41ed8068 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:28.073180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:28.079551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607deb6f-e6d0-426d-8c40-75d94a5bef12 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:28.080033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607deb6f-e6d0-426d-8c40-75d94a5bef12 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:28.080279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607deb6f-e6d0-426d-8c40-75d94a5bef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:28.157398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-607deb6f-e6d0-426d-8c40-75d94a5bef12 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:28.158622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-607deb6f-e6d0-426d-8c40-75d94a5bef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:28.225746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:28.255028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:28.403776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6162-e446-41cd-b53e-2bcf41ed8068 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:28.405341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e3ce6162-e446-41cd-b53e-2bcf41ed8068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:28.423786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:28.551072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6162-e446-41cd-b53e-2bcf41ed8068 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:28.637783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:28.663496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:28.664098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:28.664410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:28.664685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:28.666873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:28.755119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:28.756778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae8c85-98c8-47b5-83e1-04b228b92667 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:28.776133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 719/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:29.892879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef198edf-2858-4b9b-959d-6ce9e5dfb815 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:29.893428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef198edf-2858-4b9b-959d-6ce9e5dfb815 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:29.893428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef198edf-2858-4b9b-959d-6ce9e5dfb815 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:29.894752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:29.901548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa95bc-cd72-4fbb-aba7-eb709fe24e96 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:29.901993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa95bc-cd72-4fbb-aba7-eb709fe24e96 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:29.902108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa95bc-cd72-4fbb-aba7-eb709fe24e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:29.977958 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-34fa95bc-cd72-4fbb-aba7-eb709fe24e96 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:29.979837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-34fa95bc-cd72-4fbb-aba7-eb709fe24e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:30.019965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:30.078346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:30.227419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef198edf-2858-4b9b-959d-6ce9e5dfb815 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:30.229518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef198edf-2858-4b9b-959d-6ce9e5dfb815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:30.371095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef198edf-2858-4b9b-959d-6ce9e5dfb815 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:30.424255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:30.483935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:30.484493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:30.484775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:30.485048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:30.486631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:30.574704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:30.576337 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fa6e4-c1c0-4d25-bd9e-08659ae21aa2 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:30.594946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 717/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:31.700792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2d6c-06b0-4999-9dc3-9c728d20634b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:31.701326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2d6c-06b0-4999-9dc3-9c728d20634b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:31.701326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2d6c-06b0-4999-9dc3-9c728d20634b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:31.702626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:31.709086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb8c0d-0b41-49c5-a288-f8aabc9ccdf9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:31.709482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb8c0d-0b41-49c5-a288-f8aabc9ccdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:31.709636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb8c0d-0b41-49c5-a288-f8aabc9ccdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:31.786127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-83fb8c0d-0b41-49c5-a288-f8aabc9ccdf9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:31.787348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83fb8c0d-0b41-49c5-a288-f8aabc9ccdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:31.789425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:32.000507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:32.149662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2d6c-06b0-4999-9dc3-9c728d20634b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:32.151815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9ecd2d6c-06b0-4999-9dc3-9c728d20634b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:32.294832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2d6c-06b0-4999-9dc3-9c728d20634b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:32.321390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:32.408706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:32.409304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:32.409581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:32.409852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:32.411455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:32.499138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:32.500771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a361cc7-482e-4796-9dc3-5214e1cdff14 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:32.518887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 722/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:34.597596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c553a23-136c-4051-8507-3f8f45b58923 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:34.598135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c553a23-136c-4051-8507-3f8f45b58923 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:34.598135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c553a23-136c-4051-8507-3f8f45b58923 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:34.599446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:34.605294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642101f2-bc7b-46ee-9ec6-ac86d47bf36f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:34.605746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642101f2-bc7b-46ee-9ec6-ac86d47bf36f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:34.605854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642101f2-bc7b-46ee-9ec6-ac86d47bf36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:34.646208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:34.671735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-642101f2-bc7b-46ee-9ec6-ac86d47bf36f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:34.672987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-642101f2-bc7b-46ee-9ec6-ac86d47bf36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:34.770665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:35.063362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c553a23-136c-4051-8507-3f8f45b58923 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:35.065012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9c553a23-136c-4051-8507-3f8f45b58923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:35.167953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c553a23-136c-4051-8507-3f8f45b58923 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:35.198914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:35.240220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:35.240764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:35.241044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:35.241316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:35.242887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:35.349609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:35.351042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffd1be-5edf-4bb2-863e-6b2aef9e2da9 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:35.369106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 720/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:37.680339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262086-9bd6-41dc-892f-c6d98f461450 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:37.680850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262086-9bd6-41dc-892f-c6d98f461450 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:37.680850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262086-9bd6-41dc-892f-c6d98f461450 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:37.682170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:37.687402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a5c4a-dc6b-4c12-a505-15bcee10042f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:37.687801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a5c4a-dc6b-4c12-a505-15bcee10042f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:37.687952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a5c4a-dc6b-4c12-a505-15bcee10042f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:37.753867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b12a5c4a-dc6b-4c12-a505-15bcee10042f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:37.755072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b12a5c4a-dc6b-4c12-a505-15bcee10042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:37.767905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:37.851746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:38.132906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262086-9bd6-41dc-892f-c6d98f461450 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:38.135128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a1262086-9bd6-41dc-892f-c6d98f461450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:38.238310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262086-9bd6-41dc-892f-c6d98f461450 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:38.250494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:38.311412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:38.312011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:38.312317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:38.312587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:38.314170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:38.422470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:38.423902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb4fc4-2900-425d-a238-4816aacab2ca service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:38.441404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 725/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:40.992376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b528a4-68e5-4d5c-8e75-14110917dee0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:40.992870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b528a4-68e5-4d5c-8e75-14110917dee0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:40.992968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b528a4-68e5-4d5c-8e75-14110917dee0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:40.994231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:40.999487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79f30-6206-49f6-a4cb-9195c1eebd13 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:40.999894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79f30-6206-49f6-a4cb-9195c1eebd13 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:41.000042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79f30-6206-49f6-a4cb-9195c1eebd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:41.048232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:41.065587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-add79f30-6206-49f6-a4cb-9195c1eebd13 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:41.066792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-add79f30-6206-49f6-a4cb-9195c1eebd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:41.164721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:41.448215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b528a4-68e5-4d5c-8e75-14110917dee0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:41.449735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f1b528a4-68e5-4d5c-8e75-14110917dee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:41.549523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b528a4-68e5-4d5c-8e75-14110917dee0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:41.587969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:41.621357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:41.621949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:41.622216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:41.622479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:41.624044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:41.728371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:41.729752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5391b-ef2e-4857-b4ab-f7e141fdbcea service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:41.747885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 723/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:43.997620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c7407-4805-43ff-89ce-7ab98ac07270 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:43.998119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c7407-4805-43ff-89ce-7ab98ac07270 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:43.998518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c7407-4805-43ff-89ce-7ab98ac07270 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:44.000593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:44.008931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535734a-8732-4cd2-9f16-4c7ccf9ff169 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:44.009937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535734a-8732-4cd2-9f16-4c7ccf9ff169 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:44.010394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535734a-8732-4cd2-9f16-4c7ccf9ff169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:44.075192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b535734a-8732-4cd2-9f16-4c7ccf9ff169 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:44.076414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b535734a-8732-4cd2-9f16-4c7ccf9ff169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:44.096745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:44.170825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:44.461694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c7407-4805-43ff-89ce-7ab98ac07270 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:44.463747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-038c7407-4805-43ff-89ce-7ab98ac07270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:44.565103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c7407-4805-43ff-89ce-7ab98ac07270 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:44.584857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:44.637896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:44.638492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:44.638766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:44.639032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:44.640893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:44.746166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:44.747534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4c08d-6c14-4e80-b3a9-59cd28799926 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:44.765642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 728/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:46.963204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36b7f-c300-4f1b-92b9-4e286ef7816d None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:46.963776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36b7f-c300-4f1b-92b9-4e286ef7816d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:46.963776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36b7f-c300-4f1b-92b9-4e286ef7816d None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:46.965154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:46.970674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7870e-25c6-45cb-b91f-e6110f0e5bb6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:46.971104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7870e-25c6-45cb-b91f-e6110f0e5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:46.971222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7870e-25c6-45cb-b91f-e6110f0e5bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:47.036682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:47.039258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6cb7870e-25c6-45cb-b91f-e6110f0e5bb6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:47.040511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6cb7870e-25c6-45cb-b91f-e6110f0e5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:47.137240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:47.420240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36b7f-c300-4f1b-92b9-4e286ef7816d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:47.421777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-72b36b7f-c300-4f1b-92b9-4e286ef7816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:47.524166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36b7f-c300-4f1b-92b9-4e286ef7816d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:47.572383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:47.596110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:47.596735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:47.597050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:47.597392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:47.599032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:47.705616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:47.706984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7da9d-9e2a-44c0-b56b-cd4499bd7839 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:47.725135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 726/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:49.943790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee2e7-69ba-41d8-a3f0-f905c6aded54 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:49.944430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee2e7-69ba-41d8-a3f0-f905c6aded54 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:49.944430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee2e7-69ba-41d8-a3f0-f905c6aded54 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:49.945716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:49.951571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20169da2-9f1c-40bb-90b9-34563bdb28ba None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:49.951996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20169da2-9f1c-40bb-90b9-34563bdb28ba None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:49.952104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20169da2-9f1c-40bb-90b9-34563bdb28ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:50.017370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-20169da2-9f1c-40bb-90b9-34563bdb28ba None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:50.018571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-20169da2-9f1c-40bb-90b9-34563bdb28ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:50.047353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:50.115646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:50.404980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee2e7-69ba-41d8-a3f0-f905c6aded54 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:50.406522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d3bee2e7-69ba-41d8-a3f0-f905c6aded54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:50.508483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee2e7-69ba-41d8-a3f0-f905c6aded54 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:50.536759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:50.581512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:50.582117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:50.582392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:50.582658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:50.584297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:50.692321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:50.693697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac003e9-2537-4955-a5ff-d1a941b05e30 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:50.711680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 731/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:52.957789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a646-cf4c-4402-8fe3-d27c07db313a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:52.958390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a646-cf4c-4402-8fe3-d27c07db313a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:52.958390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a646-cf4c-4402-8fe3-d27c07db313a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:52.959740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:52.965825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e1f42-6737-4ac8-b0ba-1c2af17901d9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:52.966255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e1f42-6737-4ac8-b0ba-1c2af17901d9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:52.966375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e1f42-6737-4ac8-b0ba-1c2af17901d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:53.031215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-036e1f42-6737-4ac8-b0ba-1c2af17901d9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:53.032365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-036e1f42-6737-4ac8-b0ba-1c2af17901d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:53.040714 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:53.128434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:53.410360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a646-cf4c-4402-8fe3-d27c07db313a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:53.411936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ff12a646-cf4c-4402-8fe3-d27c07db313a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:53.516348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a646-cf4c-4402-8fe3-d27c07db313a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:53.572276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:53.590784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:53.591397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:53.591673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:53.591942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:53.593571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:53.701959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:53.703315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cf941-c089-4122-bbc3-b23bd9934231 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:53.721108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 729/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:55.909988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef6fd8-6598-4d84-944a-2a9bd8a4cdb8 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:55.910556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef6fd8-6598-4d84-944a-2a9bd8a4cdb8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:55.910556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef6fd8-6598-4d84-944a-2a9bd8a4cdb8 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:55.911879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:55.917201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88ead5-74f1-41f8-8bac-a96720985882 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:55.917634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88ead5-74f1-41f8-8bac-a96720985882 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:55.917764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88ead5-74f1-41f8-8bac-a96720985882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:55.984044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6d88ead5-74f1-41f8-8bac-a96720985882 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:55.985288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6d88ead5-74f1-41f8-8bac-a96720985882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:56.023209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:56.081370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:56.366899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef6fd8-6598-4d84-944a-2a9bd8a4cdb8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:56.368993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-81ef6fd8-6598-4d84-944a-2a9bd8a4cdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:56.470905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef6fd8-6598-4d84-944a-2a9bd8a4cdb8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:56.507659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:56.543580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:56.544207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:56.544485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:56.544757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:56.546322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:56.652367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:56.653743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218239b-dffd-49ab-b481-2a698f4fdd2d service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:56.671596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 734/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:39:58.815191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31363e-2943-46b8-9ccd-2997eed61f7a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:58.815834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31363e-2943-46b8-9ccd-2997eed61f7a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:58.815834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31363e-2943-46b8-9ccd-2997eed61f7a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:58.817122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:39:58.823064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1161c-6900-4052-a32f-9aef6d23e0ea None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:58.826857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1161c-6900-4052-a32f-9aef6d23e0ea None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:58.826857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1161c-6900-4052-a32f-9aef6d23e0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:58.892427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a0c1161c-6900-4052-a32f-9aef6d23e0ea None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:39:58.893696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a0c1161c-6900-4052-a32f-9aef6d23e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:58.912491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:58.990883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:39:59.280201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31363e-2943-46b8-9ccd-2997eed61f7a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:39:59.281740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b31363e-2943-46b8-9ccd-2997eed61f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:59.384384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31363e-2943-46b8-9ccd-2997eed61f7a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:39:59.449791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:39:59.456538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:39:59.457201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:39:59.457472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:39:59.457759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:39:59.459351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:39:59.566806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:39:59.568294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab29b-c745-41e9-acf2-b48455dd92c1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:39:59.586139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 732/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:01.720345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d4b2a-d73c-4032-af27-093f0ad74702 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:01.720854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d4b2a-d73c-4032-af27-093f0ad74702 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:01.720854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d4b2a-d73c-4032-af27-093f0ad74702 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:01.722220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:40:01.727663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504883c-cc04-4df7-a2ae-5b2822446b12 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:01.728056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504883c-cc04-4df7-a2ae-5b2822446b12 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:01.728239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504883c-cc04-4df7-a2ae-5b2822446b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:01.795073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9504883c-cc04-4df7-a2ae-5b2822446b12 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:01.796326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9504883c-cc04-4df7-a2ae-5b2822446b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:01.843886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:01.892600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:02.175006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d4b2a-d73c-4032-af27-093f0ad74702 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:02.176623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-051d4b2a-d73c-4032-af27-093f0ad74702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:02.279506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d4b2a-d73c-4032-af27-093f0ad74702 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:02.326694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:02.353093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:02.353693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:02.353972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:02.354274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:02.355855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:02.462256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:02.463620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d776373-d564-484f-b3b8-15351b77a302 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:02.481980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 737/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:11.890784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=22 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:12.479751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=66 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:22.411139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d28d1-35f6-4a72-a7aa-3f2fcd3ce356 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:22.411774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d28d1-35f6-4a72-a7aa-3f2fcd3ce356 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:22.411774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d28d1-35f6-4a72-a7aa-3f2fcd3ce356 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:22.413247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 734/1471] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:22.419392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884539d9-fffa-40f7-8a80-ed84cfb789ed None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:22.419831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884539d9-fffa-40f7-8a80-ed84cfb789ed None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:22.419972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884539d9-fffa-40f7-8a80-ed84cfb789ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:22.486889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-884539d9-fffa-40f7-8a80-ed84cfb789ed None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:22.488109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-884539d9-fffa-40f7-8a80-ed84cfb789ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:22.586358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 738/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:22.900843 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08313a-cefc-4c20-8e79-d335f78cfdf7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:22.901121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08313a-cefc-4c20-8e79-d335f78cfdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:22.901386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08313a-cefc-4c20-8e79-d335f78cfdf7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:22.903255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 735/1473] 10.222.0.57 () {54 vars in 961 bytes} [Wed Sep 7 14:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:22.908341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1a490-1fb5-43bb-9324-61fa336a2ca0 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:22.908631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1a490-1fb5-43bb-9324-61fa336a2ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:22.908903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1a490-1fb5-43bb-9324-61fa336a2ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:22.992249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3fa1a490-1fb5-43bb-9324-61fa336a2ca0 None None] MFA Rules not processed for user `113ccc244a814d7580d1c9f6319edf04`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:22.993459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3fa1a490-1fb5-43bb-9324-61fa336a2ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:23.016428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:23.077074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 739/1474] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Sep 7 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:23.087074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08313a-cefc-4c20-8e79-d335f78cfdf7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:23.088631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0e08313a-cefc-4c20-8e79-d335f78cfdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:23.200177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08313a-cefc-4c20-8e79-d335f78cfdf7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:23.277550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:23.278162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:23.278465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:23.278734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:23.280361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:23.334270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:23.389877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:23.391578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365feddf-c1a6-4fb9-8d7e-402c6448a4e0 service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:23.410407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 736/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:25.641738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cc55b-e7ce-436c-8a2f-832fda6f42dd None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:25.642271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cc55b-e7ce-436c-8a2f-832fda6f42dd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:25.642271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cc55b-e7ce-436c-8a2f-832fda6f42dd None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:25.643640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 740/1476] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:25.649376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034327a-52db-4fcb-9e29-3b64503b78fa None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:25.649789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034327a-52db-4fcb-9e29-3b64503b78fa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:25.649901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034327a-52db-4fcb-9e29-3b64503b78fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:25.714506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a034327a-52db-4fcb-9e29-3b64503b78fa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:25.715645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a034327a-52db-4fcb-9e29-3b64503b78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:25.811975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 737/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:26.083892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cc55b-e7ce-436c-8a2f-832fda6f42dd None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:26.085485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-819cc55b-e7ce-436c-8a2f-832fda6f42dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:26.211430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cc55b-e7ce-436c-8a2f-832fda6f42dd None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:26.259381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:26.313228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:26.313832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:26.314112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:26.314386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:26.315022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:26.316892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:40:26.316892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:40:26.316892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:40:26.318035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff15f8-6b2c-4f6c-9039-51d324fa8bef admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:26.337099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 741/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Sep 7 14:40:26 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:40:28.500862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2e95-d304-41a2-b211-e18e61990593 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:28.501420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2e95-d304-41a2-b211-e18e61990593 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:28.501420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2e95-d304-41a2-b211-e18e61990593 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:28.502756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 738/1479] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:28.508484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7bc8-47ae-47ca-a125-bf5ee9153c34 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:28.508902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7bc8-47ae-47ca-a125-bf5ee9153c34 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:28.509034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7bc8-47ae-47ca-a125-bf5ee9153c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:28.576663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cece7bc8-47ae-47ca-a125-bf5ee9153c34 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:28.577860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cece7bc8-47ae-47ca-a125-bf5ee9153c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:28.672254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 742/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:28.967083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2e95-d304-41a2-b211-e18e61990593 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:28.968750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef4a2e95-d304-41a2-b211-e18e61990593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:28.978967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:29.115101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2e95-d304-41a2-b211-e18e61990593 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:29.197801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:29.230226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:29.230750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:29.231027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:29.231297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:29.233033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:29.323509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:29.325127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9745c-ceea-49ba-888b-b4f92e0e32f4 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:29.343448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 739/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:31.647857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9234d9-96ac-46e5-b5aa-823aec20fac9 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:31.648935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9234d9-96ac-46e5-b5aa-823aec20fac9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:31.648935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9234d9-96ac-46e5-b5aa-823aec20fac9 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:31.650326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 743/1482] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:31.655816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb30c-11d6-4006-b186-fd2940351b17 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:31.656254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb30c-11d6-4006-b186-fd2940351b17 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:31.656398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb30c-11d6-4006-b186-fd2940351b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:31.721412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-526eb30c-11d6-4006-b186-fd2940351b17 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:31.722533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-526eb30c-11d6-4006-b186-fd2940351b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:31.769967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:31.819381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 740/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:32.109264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9234d9-96ac-46e5-b5aa-823aec20fac9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:32.110855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5f9234d9-96ac-46e5-b5aa-823aec20fac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:32.134500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:32.214114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9234d9-96ac-46e5-b5aa-823aec20fac9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:32.285044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:32.285641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:32.285919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:32.286187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:32.287773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:32.362673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:32.397067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:32.399043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa18f-a7d1-46c4-9368-bc8f795c52ca service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:32.417319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 744/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:34.609377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93649bf-d693-43d7-9b44-23b74a20e77d None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:34.609981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93649bf-d693-43d7-9b44-23b74a20e77d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:34.609981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93649bf-d693-43d7-9b44-23b74a20e77d None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:34.611309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 741/1485] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:34.616638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b503b-847e-4931-95e5-2ce5e835144f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:34.617101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b503b-847e-4931-95e5-2ce5e835144f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:34.617364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b503b-847e-4931-95e5-2ce5e835144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:34.684085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7f1b503b-847e-4931-95e5-2ce5e835144f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:34.685350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7f1b503b-847e-4931-95e5-2ce5e835144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:34.782098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 745/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:35.067132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93649bf-d693-43d7-9b44-23b74a20e77d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:35.068754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d93649bf-d693-43d7-9b44-23b74a20e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:35.172239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93649bf-d693-43d7-9b44-23b74a20e77d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:35.242391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:35.244177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:35.244786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:35.245065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:35.245336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:35.246919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:35.356565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:35.357996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb7bb-fb26-4be4-be1a-6dbf5789ea82 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:35.376005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 742/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:37.630348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1d061-4c0e-48f0-a4e8-dd63c85170e0 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:37.630906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1d061-4c0e-48f0-a4e8-dd63c85170e0 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:37.630906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1d061-4c0e-48f0-a4e8-dd63c85170e0 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:37.632362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 746/1488] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:37.637547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f898d-8088-42c1-bb46-28a8e37e2677 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:37.637940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f898d-8088-42c1-bb46-28a8e37e2677 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:37.638097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f898d-8088-42c1-bb46-28a8e37e2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:37.702363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ef2f898d-8088-42c1-bb46-28a8e37e2677 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:37.703497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef2f898d-8088-42c1-bb46-28a8e37e2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:37.755622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:37.800065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 743/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:38.093822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1d061-4c0e-48f0-a4e8-dd63c85170e0 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:38.095391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-39f1d061-4c0e-48f0-a4e8-dd63c85170e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:38.133574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:38.206580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1d061-4c0e-48f0-a4e8-dd63c85170e0 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:38.281386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:38.281923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:38.282201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:38.282472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:38.284066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:38.362804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:38.392057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:38.393560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f0eb9-1dca-4dd3-87bf-236e3535d28a service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:38.411803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 747/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:40.929035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa322b-495e-4188-9221-431ee869f5af None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:40.929613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa322b-495e-4188-9221-431ee869f5af None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:40.929613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa322b-495e-4188-9221-431ee869f5af None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:40.930937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 744/1491] 10.222.0.57 () {54 vars in 927 bytes} [Wed Sep 7 14:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:40:40.936121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec410ca-8783-441e-a1b8-4e70dd55e7a3 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:40.936603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec410ca-8783-441e-a1b8-4e70dd55e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:40.936712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec410ca-8783-441e-a1b8-4e70dd55e7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:41.003412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4ec410ca-8783-441e-a1b8-4e70dd55e7a3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:40:41.004646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4ec410ca-8783-441e-a1b8-4e70dd55e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:41.100960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 748/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Sep 7 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:40:41.399664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa322b-495e-4188-9221-431ee869f5af None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:40:41.401287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-98fa322b-495e-4188-9221-431ee869f5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:41.510562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa322b-495e-4188-9221-431ee869f5af None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:40:41.581435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:40:41.582027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:40:41.582301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:40:41.582565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:40:41.584130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:40:41.592964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:41.691240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:40:41.692663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c490c01-f2de-4c19-b0cf-8eb0cc7a11d8 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:40:41.711080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 745/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:40:51.099302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=82 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:40:51.708371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=50 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:02.219238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:02.219802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:02.219802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:02.296314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:42:02.297534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:02.391905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 749/1494] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:42:02.400006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:02.400528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:02.400636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:02.475329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:42:02.476494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:02.571324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 746/1495] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:42:02.575932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:02.577199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:02.588964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536a6b-d33c-4503-bfff-daa0f59670b9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:02.650044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:02.683242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:02.683842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:02.684122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:02.684419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:02.685060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:42:02.685360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:02.686377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:02.686377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:42:02.686377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:42:02.687283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:02.698450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 750/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:42:02.706718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:02.716209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d6ad4-ead4-41db-afb9-3894e3836202 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:02.777360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:02.811250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:02.811862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:02.812138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:02.812429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:02.813073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:02.814834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:02.814834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:42:02.814834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:42:02.816072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:02.926384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 747/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:42:02 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:42:02.930909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:02.932458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:02.966326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869a66-b36c-4b56-ac45-ac38efc05d48 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:03.060137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:03.060768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.061050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.061321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.061942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:03.062957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:03.062957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:42:03.062957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:42:03.064194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:03.146330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 751/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:42:02 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:42:03.151264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:03.152844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:03.216165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d067ca-506a-4e63-935b-835cd133a9a7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:03.300819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:03.315591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:03.315591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.315591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.315591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.316112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:03.316112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:03.316112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:42:03.316112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:42:03.316112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:03.328828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 748/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:42:03.332523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:03.338733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:03.345466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c292c8-0f2a-4450-96ca-cf1e2efb507d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:03.370658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:03.440865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:03.441468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.441743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.442011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.442646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:03.443666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:03.443666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:42:03.443666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:42:03.444622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9aa877-0c5c-4e69-aab7-d34d1cae649a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:03.457585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 752/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:42:03.462041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:03.467222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:03.498772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929212f-fbd0-4723-bccc-5274be6c3c32 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:03.592445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:03.593024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.593317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.593584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] PATH_INFO: `/v3/projects/b772c6883e99435da6e7be8d32126597/users/8565e04bb2164d6ab5599f7d8fd0e5d7/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.615315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b772c6883e99435da6e7be8d32126597, user_id=8565e04bb2164d6ab5599f7d8fd0e5d7, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:03.616380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:03.616380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:42:03.616380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:42:03.618957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f38469-34f0-4be7-acdd-0fdd2836383f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:03.731950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 749/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:42:03 2022] PUT /identity/v3/projects/b772c6883e99435da6e7be8d32126597/users/8565e04bb2164d6ab5599f7d8fd0e5d7/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:42:03.745334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9f7fe-c3a5-407e-89bb-7f8dc866ea7b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.745334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9f7fe-c3a5-407e-89bb-7f8dc866ea7b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.745531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9f7fe-c3a5-407e-89bb-7f8dc866ea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.830618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cbc9f7fe-c3a5-407e-89bb-7f8dc866ea7b None None] MFA Rules not processed for user `8565e04bb2164d6ab5599f7d8fd0e5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:42:03.831842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cbc9f7fe-c3a5-407e-89bb-7f8dc866ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:03.908311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 753/1502] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:42:03.922132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17085c8-0c60-4976-8332-f3d42642568f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.922419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17085c8-0c60-4976-8332-f3d42642568f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.922692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17085c8-0c60-4976-8332-f3d42642568f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.924014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 750/1503] 10.222.0.57 () {54 vars in 964 bytes} [Wed Sep 7 14:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:42:03.928789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65035a79-9176-4ced-81f5-3911cfdfbb0c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:03.929114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65035a79-9176-4ced-81f5-3911cfdfbb0c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:03.929381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65035a79-9176-4ced-81f5-3911cfdfbb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:03.960152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:04.005263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-65035a79-9176-4ced-81f5-3911cfdfbb0c None None] MFA Rules not processed for user `d83045f9a43b48d7bbcce339f7bfa5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:42:04.006459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-65035a79-9176-4ced-81f5-3911cfdfbb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:04.079856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 754/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Wed Sep 7 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:42:04.088016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17085c8-0c60-4976-8332-f3d42642568f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:04.089582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d17085c8-0c60-4976-8332-f3d42642568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:04.138786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:04.193358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17085c8-0c60-4976-8332-f3d42642568f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:04.266635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:04.267238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:04.267518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:04.267790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:04.269415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:04.366199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:04.367618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:04.371116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:04.385176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 751/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:42:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:42:04.430174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b4eb-b05a-424e-bee8-3057978bc8bb None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:04.430605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b4eb-b05a-424e-bee8-3057978bc8bb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:04.430727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b4eb-b05a-424e-bee8-3057978bc8bb None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:04.432014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 755/1506] 10.222.0.57 () {52 vars in 830 bytes} [Wed Sep 7 14:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:42:04.435668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:04.436946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:04.536468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c42f2-7204-466d-a2b9-8c86fdea4008 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:04.627270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:04.627886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:04.628176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:04.628448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] PATH_INFO: `/v3/users/8565e04bb2164d6ab5599f7d8fd0e5d7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:04.641190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8565e04bb2164d6ab5599f7d8fd0e5d7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:04.642226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:04.642226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:42:04.642226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:42:04.643469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:04.653659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:04.828239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8565e04bb2164d6ab5599f7d8fd0e5d7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:04.828689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8565e04bb2164d6ab5599f7d8fd0e5d7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:04.831490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8565e04bb2164d6ab5599f7d8fd0e5d7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:04.873871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8565e04bb2164d6ab5599f7d8fd0e5d7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:04.888248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8565e04bb2164d6ab5599f7d8fd0e5d7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:04.916892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 752/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:42:04 2022] DELETE /identity/v3/users/8565e04bb2164d6ab5599f7d8fd0e5d7 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:42:04.921415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:04.921698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:04.921967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:04.978402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:42:04.979598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:05.053136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:05.064971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 756/1508] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:42:05.075890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:05.077065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:05.181156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5d953-4837-44c2-a0ca-66d2f5466dad admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:05.235826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:05.254331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:05.254941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:05.255222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:05.255494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:05.257186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:05.368184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:05.369546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3cb5-d741-48a7-ac04-134a337df797 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:05.387928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 753/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:42:05.922646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:42:05.924620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:42:05.939235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc1e53-c82d-4eb9-8abb-4a14443c023b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:42:06.034717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:42:06.035350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:42:06.035630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:42:06.035902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] PATH_INFO: `/v3/projects/b772c6883e99435da6e7be8d32126597` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:42:06.046116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b772c6883e99435da6e7be8d32126597)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:42:06.047151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:42:06.047151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:42:06.047151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:42:06.048396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:42:06.323181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b772c6883e99435da6e7be8d32126597', 'request_id': 'req-8b63431c-9b88-4c70-ae37-b723f31234e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:06.394062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b772c6883e99435da6e7be8d32126597', 'request_id': 'req-8b63431c-9b88-4c70-ae37-b723f31234e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:06.401191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b772c6883e99435da6e7be8d32126597', 'request_id': 'req-8b63431c-9b88-4c70-ae37-b723f31234e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:06.431687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] The token cache is being invalidate because project b772c6883e99435da6e7be8d32126597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:42:06.431973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:06.432523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b63431c-9b88-4c70-ae37-b723f31234e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:42:06.433829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 757/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:42:05 2022] DELETE /identity/v3/projects/b772c6883e99435da6e7be8d32126597 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:42:15.385706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=62 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:42:16.429677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:43:35.565213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0b4e-3ee3-4d37-bbf2-ec2b40b534cc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:35.565771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0b4e-3ee3-4d37-bbf2-ec2b40b534cc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:35.565771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0b4e-3ee3-4d37-bbf2-ec2b40b534cc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:35.567089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:43:35.571904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b521b-cb48-4126-9c89-022638298683 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:35.572392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b521b-cb48-4126-9c89-022638298683 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:35.572525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b521b-cb48-4126-9c89-022638298683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:35.652719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-c69b521b-cb48-4126-9c89-022638298683 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:35.653931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c69b521b-cb48-4126-9c89-022638298683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:35.755870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:43:35.916129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b0b4e-3ee3-4d37-bbf2-ec2b40b534cc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:35.922115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-601b0b4e-3ee3-4d37-bbf2-ec2b40b534cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:36.044209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b0b4e-3ee3-4d37-bbf2-ec2b40b534cc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:36.049664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57871b4-3d44-4d8a-a105-6f2a8a4086b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:36.050268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57871b4-3d44-4d8a-a105-6f2a8a4086b2 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:36.050549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57871b4-3d44-4d8a-a105-6f2a8a4086b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:36.050821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57871b4-3d44-4d8a-a105-6f2a8a4086b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:36.058748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57871b4-3d44-4d8a-a105-6f2a8a4086b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:36.060358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57871b4-3d44-4d8a-a105-6f2a8a4086b2 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:36.069594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 755/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:43:37.089011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac6f26-cb8e-4df7-bcfd-c387701cc4e7 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:37.089562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac6f26-cb8e-4df7-bcfd-c387701cc4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:37.089562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac6f26-cb8e-4df7-bcfd-c387701cc4e7 None None] PATH_INFO: `/v3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:37.090847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 759/1514] 10.222.0.39 () {54 vars in 925 bytes} [Wed Sep 7 14:43:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:43:37.097554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e2d0f-4779-4f4c-ace9-3de4b81a21dd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:37.097955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e2d0f-4779-4f4c-ace9-3de4b81a21dd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:37.098262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e2d0f-4779-4f4c-ace9-3de4b81a21dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:37.158316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-848e2d0f-4779-4f4c-ace9-3de4b81a21dd None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:37.159172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7eaa3-69d1-4574-9ed8-22c2ddbb5541 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:37.159452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7eaa3-69d1-4574-9ed8-22c2ddbb5541 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:37.159568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-848e2d0f-4779-4f4c-ace9-3de4b81a21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:37.159721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7eaa3-69d1-4574-9ed8-22c2ddbb5541 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:37.161109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 760/1515] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:43:37.167456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:37.167730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:37.167997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:37.188076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:37.189291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:37.196971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 761/1516] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:43:37.229092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:43:37.234196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 756/1517] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Sep 7 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:43:37.347976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:37.360840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a7e71-f01b-4654-b3b6-35788a824e6f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:37.366284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a80d41-6131-41fe-a375-e99754806327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:37.366875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a80d41-6131-41fe-a375-e99754806327 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:37.367152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a80d41-6131-41fe-a375-e99754806327 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:37.367421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a80d41-6131-41fe-a375-e99754806327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:37.375961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a80d41-6131-41fe-a375-e99754806327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:37.377579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a80d41-6131-41fe-a375-e99754806327 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:37.386961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 762/1518] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:43:40.951740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215a8e-aa77-4c1d-8666-ed7ef9af66cf None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:40.952189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215a8e-aa77-4c1d-8666-ed7ef9af66cf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:40.952335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215a8e-aa77-4c1d-8666-ed7ef9af66cf None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:40.953676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 757/1519] 10.222.0.57 () {54 vars in 961 bytes} [Wed Sep 7 14:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:43:40.958513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d64e-6ca8-46a2-8f3f-c67b057d25f3 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:40.958941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d64e-6ca8-46a2-8f3f-c67b057d25f3 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:40.959065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d64e-6ca8-46a2-8f3f-c67b057d25f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:40.980252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f273d64e-6ca8-46a2-8f3f-c67b057d25f3 None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:40.981448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f273d64e-6ca8-46a2-8f3f-c67b057d25f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:40.990036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 763/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Sep 7 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:43:40.997811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215a8e-aa77-4c1d-8666-ed7ef9af66cf None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:41.011384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215a8e-aa77-4c1d-8666-ed7ef9af66cf None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:41.016603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24568f80-c21a-44ef-b9a6-cffd77c6aebf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:41.017192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568f80-c21a-44ef-b9a6-cffd77c6aebf service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:41.017468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568f80-c21a-44ef-b9a6-cffd77c6aebf service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:41.017737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568f80-c21a-44ef-b9a6-cffd77c6aebf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:41.026541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24568f80-c21a-44ef-b9a6-cffd77c6aebf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:41.028113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24568f80-c21a-44ef-b9a6-cffd77c6aebf service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:41.037907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 758/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:43:48.943823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722ccfa-3b31-4061-bd18-1581771870c4 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:48.944405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722ccfa-3b31-4061-bd18-1581771870c4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:48.944405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722ccfa-3b31-4061-bd18-1581771870c4 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:48.945693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:43:48.951859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3c3f6-6004-4075-9358-79eaa5d8ee12 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:48.952344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3c3f6-6004-4075-9358-79eaa5d8ee12 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:48.952463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3c3f6-6004-4075-9358-79eaa5d8ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:48.972877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-51d3c3f6-6004-4075-9358-79eaa5d8ee12 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:48.974150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-51d3c3f6-6004-4075-9358-79eaa5d8ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:48.983104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:43:49.133891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722ccfa-3b31-4061-bd18-1581771870c4 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:49.148677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722ccfa-3b31-4061-bd18-1581771870c4 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:49.154831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcab0ce-ded4-442f-91a2-bb4e6fea252c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:49.155419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcab0ce-ded4-442f-91a2-bb4e6fea252c service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:49.155689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcab0ce-ded4-442f-91a2-bb4e6fea252c service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:49.155953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcab0ce-ded4-442f-91a2-bb4e6fea252c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:49.164378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcab0ce-ded4-442f-91a2-bb4e6fea252c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:49.170111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcab0ce-ded4-442f-91a2-bb4e6fea252c service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:49.175773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 765/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:43:50.283370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d509bf-8910-40fa-97c4-8693c481df4d None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:50.283908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d509bf-8910-40fa-97c4-8693c481df4d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:50.283908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d509bf-8910-40fa-97c4-8693c481df4d None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:50.285221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:43:50.294332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00e1f2-11ac-4e62-b850-7967c37369f1 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:50.294792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00e1f2-11ac-4e62-b850-7967c37369f1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:50.294893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00e1f2-11ac-4e62-b850-7967c37369f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:50.318034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ce00e1f2-11ac-4e62-b850-7967c37369f1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:50.319247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ce00e1f2-11ac-4e62-b850-7967c37369f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:50.327260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:43:50.483206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d509bf-8910-40fa-97c4-8693c481df4d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:50.497206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d509bf-8910-40fa-97c4-8693c481df4d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:50.502480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8744f78-d71c-4b5d-b170-7c6cbab2d90d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:50.503062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744f78-d71c-4b5d-b170-7c6cbab2d90d service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:50.503341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744f78-d71c-4b5d-b170-7c6cbab2d90d service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:50.503600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744f78-d71c-4b5d-b170-7c6cbab2d90d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:50.511419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8744f78-d71c-4b5d-b170-7c6cbab2d90d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:50.513091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8744f78-d71c-4b5d-b170-7c6cbab2d90d service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:50.527136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 761/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:43:54.698476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392d5e3-a478-4b69-ad09-78b0267493ca None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:54.699132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392d5e3-a478-4b69-ad09-78b0267493ca None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:54.699132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392d5e3-a478-4b69-ad09-78b0267493ca None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:54.700400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:43:54.705529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179f296-813e-4e1d-874f-349c94a1b18e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:54.705969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179f296-813e-4e1d-874f-349c94a1b18e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:54.706078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179f296-813e-4e1d-874f-349c94a1b18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:54.726924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1179f296-813e-4e1d-874f-349c94a1b18e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:54.728980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1179f296-813e-4e1d-874f-349c94a1b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:54.737168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:43:55.028623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392d5e3-a478-4b69-ad09-78b0267493ca None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:55.030303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f392d5e3-a478-4b69-ad09-78b0267493ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:55.056207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:43:55.107590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392d5e3-a478-4b69-ad09-78b0267493ca None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:55.111152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27475bb-ee8d-4a4d-b649-779b9ded2431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:55.111748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27475bb-ee8d-4a4d-b649-779b9ded2431 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:55.112024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27475bb-ee8d-4a4d-b649-779b9ded2431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:55.112325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27475bb-ee8d-4a4d-b649-779b9ded2431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:55.121004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27475bb-ee8d-4a4d-b649-779b9ded2431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:55.122436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27475bb-ee8d-4a4d-b649-779b9ded2431 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:55.131919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 768/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:43:57.256530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc94bba-bd10-4a22-9062-27e45a252820 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:57.257059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc94bba-bd10-4a22-9062-27e45a252820 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:57.257059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc94bba-bd10-4a22-9062-27e45a252820 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:57.258367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:43:57.264318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e96e2-5268-49af-9375-333ce3b78828 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:57.264753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e96e2-5268-49af-9375-333ce3b78828 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:57.264856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e96e2-5268-49af-9375-333ce3b78828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:57.284069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ba9e96e2-5268-49af-9375-333ce3b78828 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:43:57.285290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ba9e96e2-5268-49af-9375-333ce3b78828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:43:57.292975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:43:57.584725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc94bba-bd10-4a22-9062-27e45a252820 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:43:57.595624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc94bba-bd10-4a22-9062-27e45a252820 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:43:57.599150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d04e4-d1af-4d09-9028-dfce3d454c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:43:57.599748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d04e4-d1af-4d09-9028-dfce3d454c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:43:57.600027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d04e4-d1af-4d09-9028-dfce3d454c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:43:57.600351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d04e4-d1af-4d09-9028-dfce3d454c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:43:57.609769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d04e4-d1af-4d09-9028-dfce3d454c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:43:57.611181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d04e4-d1af-4d09-9028-dfce3d454c5b service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:43:57.621451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 764/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:44:01.296356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327289c2-80ac-4fe5-ad46-9579b343f805 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:44:01.296896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327289c2-80ac-4fe5-ad46-9579b343f805 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:44:01.296896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327289c2-80ac-4fe5-ad46-9579b343f805 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:44:01.298225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:44:01.303371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcf3c1-4bd2-4396-919d-f587d62280a8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:44:01.303769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcf3c1-4bd2-4396-919d-f587d62280a8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:44:01.303916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcf3c1-4bd2-4396-919d-f587d62280a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:44:01.323837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9afcf3c1-4bd2-4396-919d-f587d62280a8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:44:01.325065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9afcf3c1-4bd2-4396-919d-f587d62280a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:44:01.332942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:44:01.637352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327289c2-80ac-4fe5-ad46-9579b343f805 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:44:01.639037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-327289c2-80ac-4fe5-ad46-9579b343f805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:44:01.714776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327289c2-80ac-4fe5-ad46-9579b343f805 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:44:01.718349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd29ab-8f52-48f5-8ebd-f0059f4e626a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:44:01.718933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd29ab-8f52-48f5-8ebd-f0059f4e626a service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:44:01.719207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd29ab-8f52-48f5-8ebd-f0059f4e626a service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:44:01.719471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd29ab-8f52-48f5-8ebd-f0059f4e626a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:44:01.728177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd29ab-8f52-48f5-8ebd-f0059f4e626a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:44:01.729622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd29ab-8f52-48f5-8ebd-f0059f4e626a service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:44:01.738749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 771/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:44:03.874627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b951a-98c5-4deb-bd0e-e5c432372812 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:44:03.875180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b951a-98c5-4deb-bd0e-e5c432372812 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:44:03.875180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b951a-98c5-4deb-bd0e-e5c432372812 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:44:03.876521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 14:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:44:03.881800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd1711-3d71-48d6-b832-fe1c7a7ddb3d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:44:03.882212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd1711-3d71-48d6-b832-fe1c7a7ddb3d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:44:03.882324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd1711-3d71-48d6-b832-fe1c7a7ddb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:44:03.902382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-fefd1711-3d71-48d6-b832-fe1c7a7ddb3d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:44:03.903580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fefd1711-3d71-48d6-b832-fe1c7a7ddb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:44:03.911358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:44:04.221510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b951a-98c5-4deb-bd0e-e5c432372812 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:44:04.223205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-014b951a-98c5-4deb-bd0e-e5c432372812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:44:04.236659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b951a-98c5-4deb-bd0e-e5c432372812 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:44:04.240126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8cdb7-813a-4b9f-a497-946c006ea289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:44:04.240744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8cdb7-813a-4b9f-a497-946c006ea289 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:44:04.241019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8cdb7-813a-4b9f-a497-946c006ea289 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:44:04.241289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8cdb7-813a-4b9f-a497-946c006ea289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:44:04.250024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8cdb7-813a-4b9f-a497-946c006ea289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:44:04.251425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8cdb7-813a-4b9f-a497-946c006ea289 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:44:04.261127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 767/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:44:13.897107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=66 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:44:14.256674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=34 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:29.582553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.583226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.583334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.618068 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:29.619303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:29.632085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 773/1540] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:45:29.642349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.643468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.644087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.681767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:29.683438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:29.693857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 768/1541] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:29.699043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:29.713870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806627-2a96-4f04-a692-5aad7aaca841 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:29.724930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:29.725569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.725846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.726111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.727422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:29.727722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:29.729490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6836a-c16e-49c5-aeca-5f1ed7749968 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:29.749596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 774/1542] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:29.758619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:29.775074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876aacd-3606-4e34-b27f-d036bbcf607b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:29.783981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5e724-9c88-450d-9ed7-4bd8a585e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:29.784373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e724-9c88-450d-9ed7-4bd8a585e73f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.784853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e724-9c88-450d-9ed7-4bd8a585e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.785332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e724-9c88-450d-9ed7-4bd8a585e73f admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.786171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5e724-9c88-450d-9ed7-4bd8a585e73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:29.789162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5e724-9c88-450d-9ed7-4bd8a585e73f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:29.914304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868752b8-1954-49ce-b23b-e327e122600d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.914502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868752b8-1954-49ce-b23b-e327e122600d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.914777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868752b8-1954-49ce-b23b-e327e122600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.942124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-868752b8-1954-49ce-b23b-e327e122600d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:29.943338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-868752b8-1954-49ce-b23b-e327e122600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:29.955686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 775/1543] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:29.957863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d7511-267b-47cd-8f13-92bcf9ab99da None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.957863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d7511-267b-47cd-8f13-92bcf9ab99da None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.957863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d7511-267b-47cd-8f13-92bcf9ab99da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.965080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 769/1544] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:29 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:29.966766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94401a-649d-49d1-bc4f-46c118593352 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:29.967283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94401a-649d-49d1-bc4f-46c118593352 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:29.967836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94401a-649d-49d1-bc4f-46c118593352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:29.994197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cb7d7511-267b-47cd-8f13-92bcf9ab99da None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:29.996204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cb7d7511-267b-47cd-8f13-92bcf9ab99da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.010432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1d94401a-649d-49d1-bc4f-46c118593352 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.012123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1d94401a-649d-49d1-bc4f-46c118593352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.076808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 776/1545] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.078203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.078493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.078768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.088798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 770/1546] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.093875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8cac-3482-4cef-8eeb-46da0f99265c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.093875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8cac-3482-4cef-8eeb-46da0f99265c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.093875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8cac-3482-4cef-8eeb-46da0f99265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.101594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.102801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.112295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 777/1547] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.117941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-edee8cac-3482-4cef-8eeb-46da0f99265c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.118061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.118162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-edee8cac-3482-4cef-8eeb-46da0f99265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.124264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 771/1548] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.124264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f559a40-00c2-4008-962f-9c2c1e2f52db None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.124264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f559a40-00c2-4008-962f-9c2c1e2f52db None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.124264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f559a40-00c2-4008-962f-9c2c1e2f52db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.124747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983dc6-81ce-44f2-8ec5-fc7ed260f15f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.128403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4ada-dafe-45a3-878a-1a897a2771d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.129003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4ada-dafe-45a3-878a-1a897a2771d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.129281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4ada-dafe-45a3-878a-1a897a2771d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.129550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4ada-dafe-45a3-878a-1a897a2771d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.130176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c4ada-dafe-45a3-878a-1a897a2771d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.132217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c4ada-dafe-45a3-878a-1a897a2771d1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.142664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4f559a40-00c2-4008-962f-9c2c1e2f52db None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.143953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4f559a40-00c2-4008-962f-9c2c1e2f52db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.151914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 772/1549] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.153106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74271b8c-22ae-451c-aa24-4439230f1714 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.153382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74271b8c-22ae-451c-aa24-4439230f1714 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.153745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74271b8c-22ae-451c-aa24-4439230f1714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.173761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-74271b8c-22ae-451c-aa24-4439230f1714 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.174969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-74271b8c-22ae-451c-aa24-4439230f1714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.183343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 773/1550] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.184686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d32d-53de-4f18-af2c-c72f2b2791b8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.185028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d32d-53de-4f18-af2c-c72f2b2791b8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.185506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d32d-53de-4f18-af2c-c72f2b2791b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.208493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9e88d32d-53de-4f18-af2c-c72f2b2791b8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.209797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9e88d32d-53de-4f18-af2c-c72f2b2791b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.217583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 774/1551] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.218811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14810-9e1b-43b5-adf2-5ab9fd8239f3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.219087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14810-9e1b-43b5-adf2-5ab9fd8239f3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.219356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14810-9e1b-43b5-adf2-5ab9fd8239f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.238703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-eea14810-9e1b-43b5-adf2-5ab9fd8239f3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.239896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eea14810-9e1b-43b5-adf2-5ab9fd8239f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.248365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 775/1552] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.258550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.258550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.258550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.258885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 778/1553] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:30.258885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccb8e-fc91-4918-970c-f37826c04ac7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.258885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccb8e-fc91-4918-970c-f37826c04ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.258885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccb8e-fc91-4918-970c-f37826c04ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.270823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.272022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.277103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8c4ccb8e-fc91-4918-970c-f37826c04ac7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.285682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8c4ccb8e-fc91-4918-970c-f37826c04ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.285814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 776/1554] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.285814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.290611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 779/1555] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.291577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0a3a3-048f-4339-b380-8b9e643fe4b0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.291737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38fcdc-3ae4-465b-90dc-601ff88fae6d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.292013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38fcdc-3ae4-465b-90dc-601ff88fae6d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.292310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38fcdc-3ae4-465b-90dc-601ff88fae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.298340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.299520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.299520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.299759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.300464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.301647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.302709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:45:30.302709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:45:30.302709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:45:30.303659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fd27e-66ca-4752-b7e6-53d3b6bf2af3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.312942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ca38fcdc-3ae4-465b-90dc-601ff88fae6d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.314168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ca38fcdc-3ae4-465b-90dc-601ff88fae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.316531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 777/1556] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.322023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.322023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.322023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.322888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 780/1557] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.324344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.327948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.328283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.339835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.341083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.348885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:30.349353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 778/1558] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.350155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.353302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:30.359705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7737d-13cf-45aa-966d-861da29278ff None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.361996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 781/1559] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:30.362772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.364598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.365547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.365660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.365935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.366608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.366910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.369041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.372458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811227a4-c219-429e-bdfa-0e6b4f175392 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.376945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.377525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.377798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.378061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.378681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.380291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.387778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 779/1560] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.387778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.390766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 782/1561] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:30.391806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.399406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29acdd5-87d1-4f33-b849-4c86de5afe4f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.401469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a21a7-176d-4dd3-b10b-9863e4d4ea64 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.404046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.404797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.405083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.405356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.405993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.406133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.406315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.406566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.406875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.407141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.407772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.407887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.408077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.409752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.421015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 780/1562] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.421786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.429640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 783/1563] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.429640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.431770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948043a-f8bb-44c4-8961-61f03b686b3a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.436392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.436995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.437275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.437544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.438178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.439509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3778-56e5-4787-9ad2-49ad40d7b29e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.440102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.444001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.444667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.444946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.445365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.446087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.446476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.448057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.459601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 784/1564] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.460494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.471560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c143648-10a6-4d02-be63-b6028bf8c21f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.476290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.477476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.477756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.478076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.478701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.479000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.480597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.492258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 785/1565] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.493065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.502585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069b86-6ff1-45c8-9700-7092aad2ff2c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.507027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 781/1566] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:30.507269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.512511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.512633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.512737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.512737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.512737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:30.512737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.512737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.524423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 786/1567] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:30.525284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.528985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:30.534902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc584f-389b-42d1-83ab-b1b39cc3b6d5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.563426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:30.578395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc402d-aeaa-4d48-895e-9c18ee982988 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.583094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.583704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.583982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.584277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.584908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.587577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.598018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.599294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.599746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.600016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.600695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.602490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.612234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 787/1568] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:30.613155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.623533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f239f6-3d80-46ad-85f1-876056771183 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.628589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.629259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.629646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.630016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.630748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.631904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:45:30.631904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:45:30.631904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:45:30.634011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.680332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 782/1569] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:30.681158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.742160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecd2a6-c536-4bae-94d0-ac7994ec292b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.746863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.747463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.747742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.748015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.748684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.750655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.793943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 788/1570] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 405 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:30.794910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.820996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 783/1571] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 391 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:30.821921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.858641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d37882-903d-4112-9270-a58f4cdc0578 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.886845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6f89-2c71-477e-a7d4-71492d1b9198 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:30.891925 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.892567 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.892844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.893171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.893804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.895709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.921364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:30.921973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:30.923273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:30.923273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:30.923273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:30.925115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:30.976201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 784/1572] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:30.977922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:30.990134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:30.999595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 789/1573] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.001903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.033461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:31.040877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3435c2-d343-46b6-86f2-bb58c890b32e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.061059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694567b8-fffd-4d85-b273-7faba29fd222 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.065649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.066253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.066529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.066798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.067424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.069376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.098229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.098838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.099114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.099381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.100003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.101038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:45:31.101038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:45:31.101038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:45:31.102854 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.153270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 790/1574] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/projects => generated 389 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.154089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.172438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 785/1575] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:30 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.172438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.180191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ac302-a721-4182-aaec-b15eb66baf71 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.224112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097a93-2f55-4b90-9a17-cf20e25949dc admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.230207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.230881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.231162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.231435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] PATH_INFO: `/v3/projects/a4440e890c4a498d8ae85b2f2eefde6b/users/514ba228bc314c18ad2f3bc3bc5c78ca/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.244357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.244969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.245247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.245514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.245761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4440e890c4a498d8ae85b2f2eefde6b, user_id=514ba228bc314c18ad2f3bc3bc5c78ca, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.246139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.246811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:45:31.246811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:45:31.246811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:45:31.248041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.248799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.323395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 791/1576] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:31 2022] PUT /identity/v3/projects/a4440e890c4a498d8ae85b2f2eefde6b/users/514ba228bc314c18ad2f3bc3bc5c78ca/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:31.324244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.337326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 786/1577] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.338143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.350251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b38b9-f1e7-4ae3-abca-0af2bb540f71 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.390476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:31.410343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ed7ba-2ca0-42bc-b6b3-cc50c241196f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.415736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.416150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.416386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.416663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.416833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.416933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.417136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.417409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.417541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.418029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.419458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.419940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.482820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 787/1578] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.483533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.492859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de868ea6-8e36-4f83-8934-55c2b7433cd5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.497476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.498050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.498361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.498593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.499206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.502091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 792/1579] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.510437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.510638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.514375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3082980-2312-4878-a4ed-fe0f3532085a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.520750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.521773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.522280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.522774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.524471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.526701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.586780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 788/1580] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.587770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.597502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1102e8f-4960-421d-9fec-f6f0aabad3e4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.602036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.602915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.603067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.603356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.603985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.605952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.614670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.615497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.625564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b841c5-f524-4791-b223-873ace3aa71d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.630216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195f977-2fb5-4dd9-85a0-3561c0f24159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.630784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f977-2fb5-4dd9-85a0-3561c0f24159 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.631060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f977-2fb5-4dd9-85a0-3561c0f24159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.631327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f977-2fb5-4dd9-85a0-3561c0f24159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.631946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195f977-2fb5-4dd9-85a0-3561c0f24159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.634235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195f977-2fb5-4dd9-85a0-3561c0f24159 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.645951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 794/1582] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.647190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.647467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.647735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.654586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:31.674204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] MFA Rules not processed for user `514ba228bc314c18ad2f3bc3bc5c78ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:31.675377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:31.704347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 789/1583] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.704347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.719224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b154129-39e0-4f28-83b7-3a3cecf1cfe7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.724039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.724684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.727590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.727590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.727590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.727590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.738399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 790/1584] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.739334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.744131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 795/1585] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:31.745134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.749806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53391b-a324-4383-b305-8c02b1216c4c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.754480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.755072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.761618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.761618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.761847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a21323-2e98-4546-a9ac-ac70a45a25a4 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.761952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.761952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.762116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.762116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.762306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.762472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.763136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.764786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.768529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 791/1586] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.769307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.777952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 796/1587] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.777952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.781392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc39a-75d3-4b9b-8801-c5e8b5cb03a6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.786072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c1f1-43d6-415b-9ebc-35e6289f4ac9 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.786212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.786823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.787102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.787375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.788043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.789735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.790598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.791177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.791452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.791719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.792373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.793949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.799979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 792/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.800840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.804200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 797/1589] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.805618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.812711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ce08-ff8b-4c47-bd12-730901a68010 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.815243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee950-40a3-4cea-9088-48ec6ff44153 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.817438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.818225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.818626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.818901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.819526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.819941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.821193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.821468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.821733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.822361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.822656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.823931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.833293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 793/1590] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.834075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.834850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 798/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.835623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.845270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d16c8a-9338-4927-8e78-012e5aab54d3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.845523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c32bb1-8ed4-4d45-8dd4-8dfabeb71e72 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.849874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.850477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.850757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.851036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.851219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.851669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.851825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.852098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.852398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.853028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.853609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.854593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.867771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 794/1592] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.869322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.881037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766507-7c49-4ae1-8cdb-cdfedb3dbd2a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.886483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.887077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.887353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.887619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.888320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.889896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.902849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 795/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.904531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.913623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e14a-2af9-4b4a-9439-a137fb30495b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.918170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.919717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.919717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.919717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.920028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.921383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.931723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 796/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.932493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.941669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516366ca-54c5-4ca1-948a-e6fb70a6c9bf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:31.946793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:31.947356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:31.947630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:31.947895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:31.948420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 799/1595] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:31 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:31.948534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:31.949286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.950084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:31.960405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 797/1596] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:31.961737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:31.977718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dd848-d43d-4ffa-b26b-e8e85f0021ac admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.012915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba9174-5eef-42d6-b340-f727c25ab5a4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.017747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.018348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.018661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.018933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.019576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.021244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.031199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 800/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:32.032046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.037190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.037808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.038092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.038367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.039587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.041235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.041874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a13b64-d3e4-40fe-9b13-80c203d92c60 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.046663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:32.046919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.047551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.047833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.048109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] PATH_INFO: `/v3/projects/e03d56845a594ecf9c320b2d70a633ca/users/8eb5509df78d45acbebc332a889251dc/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.052026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 798/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:32.057370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.063057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd56a4-557a-46b0-9f6a-ed0790f677c9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.065290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03d56845a594ecf9c320b2d70a633ca, user_id=8eb5509df78d45acbebc332a889251dc, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.067753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.067961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.068436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.068770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.069051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] PATH_INFO: `/v3/projects/f669e473264f4f52b07819cc9ca21576/users/d261369de0bf488ba4645da90e41ade3/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.085042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f669e473264f4f52b07819cc9ca21576, user_id=d261369de0bf488ba4645da90e41ade3, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.087715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.127735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 801/1599] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/e03d56845a594ecf9c320b2d70a633ca/users/8eb5509df78d45acbebc332a889251dc/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.129108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.139054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:32.152131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 799/1600] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/f669e473264f4f52b07819cc9ca21576/users/d261369de0bf488ba4645da90e41ade3/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.153288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.191908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747b9ae-2779-4463-b4bb-e63d846fcaf4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.225673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd5017-d215-422c-a792-40259533b5cd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.231093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.235324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.235324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.235324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] PATH_INFO: `/v3/projects/22099e81fa2b4333ba3d9806ae2c4b5b/users/7f637cb3111a49c7b6c7d1514ab53719/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.247506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] RBAC: Authorizing `identity:create_grant(project_id=22099e81fa2b4333ba3d9806ae2c4b5b, user_id=7f637cb3111a49c7b6c7d1514ab53719, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.249030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.249689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.249968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.250170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.250277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] PATH_INFO: `/v3/projects/6869744faba14c0bb69be840cea600f0/users/9d8116d648d64a62af52d9db6a607e13/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.264925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6869744faba14c0bb69be840cea600f0, user_id=9d8116d648d64a62af52d9db6a607e13, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.267825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.358026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 800/1601] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/22099e81fa2b4333ba3d9806ae2c4b5b/users/7f637cb3111a49c7b6c7d1514ab53719/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.358984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.373541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 802/1602] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/6869744faba14c0bb69be840cea600f0/users/9d8116d648d64a62af52d9db6a607e13/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.374354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.430618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffbf9b-f3ff-4ee2-bfd9-4025960e87db admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.439647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44544f6b-34f0-4927-9544-410d4e551aef admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.444389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.445040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.445569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.445846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] PATH_INFO: `/v3/projects/ee79ffed913244e18fec0abd00da8fa7/users/bbfb8ac1b9df4d968fde391a5ec6db4c/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.460390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee79ffed913244e18fec0abd00da8fa7, user_id=bbfb8ac1b9df4d968fde391a5ec6db4c, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.463588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.486009 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.486710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.486985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.487256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.487899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.489932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.500686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:32.534408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 803/1603] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/ee79ffed913244e18fec0abd00da8fa7/users/bbfb8ac1b9df4d968fde391a5ec6db4c/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.535143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.564750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 801/1604] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:32.564750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.569285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:32.573755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1673e-41b5-4747-882a-0cf0a5cea491 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.605809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ea89a-7250-4232-9d8f-681813b76240 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.611293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302bdfb-3a9a-4b59-8de7-c3d23ef845f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.612034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302bdfb-3a9a-4b59-8de7-c3d23ef845f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.612343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302bdfb-3a9a-4b59-8de7-c3d23ef845f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.612616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302bdfb-3a9a-4b59-8de7-c3d23ef845f7 admin admin] PATH_INFO: `/v3/projects/d95dab0bcea84d42b8b92ce81606e2ec/users/9f46e69140e347669da323ad08edd7e5/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.629165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302bdfb-3a9a-4b59-8de7-c3d23ef845f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95dab0bcea84d42b8b92ce81606e2ec, user_id=9f46e69140e347669da323ad08edd7e5, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.631256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7b02d-9dbb-4bb6-b463-ce06557a23f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.631924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7b02d-9dbb-4bb6-b463-ce06557a23f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.632031 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302bdfb-3a9a-4b59-8de7-c3d23ef845f7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.632234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7b02d-9dbb-4bb6-b463-ce06557a23f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.632512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7b02d-9dbb-4bb6-b463-ce06557a23f8 admin admin] PATH_INFO: `/v3/projects/fa5b6b66febc4070a509feee0b0838f6/users/eeab924bf7a646779b48f57f14710e05/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.647145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7b02d-9dbb-4bb6-b463-ce06557a23f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5b6b66febc4070a509feee0b0838f6, user_id=eeab924bf7a646779b48f57f14710e05, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.650371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7b02d-9dbb-4bb6-b463-ce06557a23f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.716969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 804/1605] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/d95dab0bcea84d42b8b92ce81606e2ec/users/9f46e69140e347669da323ad08edd7e5/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.718228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751c8eb-4609-4219-a258-f25e5bb33efa None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.718511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751c8eb-4609-4219-a258-f25e5bb33efa None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.718782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751c8eb-4609-4219-a258-f25e5bb33efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.737359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f751c8eb-4609-4219-a258-f25e5bb33efa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.738548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f751c8eb-4609-4219-a258-f25e5bb33efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.751364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 802/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:32 2022] PUT /identity/v3/projects/fa5b6b66febc4070a509feee0b0838f6/users/eeab924bf7a646779b48f57f14710e05/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:32.755145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f085b7-8910-4557-b102-d2812b5e7bf2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.755145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f085b7-8910-4557-b102-d2812b5e7bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.755145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f085b7-8910-4557-b102-d2812b5e7bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.784768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-78f085b7-8910-4557-b102-d2812b5e7bf2 None None] MFA Rules not processed for user `d261369de0bf488ba4645da90e41ade3`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.785930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-78f085b7-8910-4557-b102-d2812b5e7bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.799723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 805/1607] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.801133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f326d-26c6-48f4-a0f5-e56626ff4c2c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.801402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f326d-26c6-48f4-a0f5-e56626ff4c2c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.801676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f326d-26c6-48f4-a0f5-e56626ff4c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.822385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-566f326d-26c6-48f4-a0f5-e56626ff4c2c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.823592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-566f326d-26c6-48f4-a0f5-e56626ff4c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.856350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 803/1608] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.858028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.858322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.858619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.877608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.888573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.888716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 806/1609] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.888716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.888716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.889066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 804/1610] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.889154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.889248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.900611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065085c-774b-4d30-bf5c-5bccd32f2363 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:32.905969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ea18b-a2d4-42fa-9f58-cda226e13c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:32.906666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea18b-a2d4-42fa-9f58-cda226e13c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.906941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea18b-a2d4-42fa-9f58-cda226e13c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.907218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea18b-a2d4-42fa-9f58-cda226e13c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.907862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea18b-a2d4-42fa-9f58-cda226e13c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:32.909496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea18b-a2d4-42fa-9f58-cda226e13c2c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:32.917819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] MFA Rules not processed for user `bbfb8ac1b9df4d968fde391a5ec6db4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.919024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.920125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 805/1611] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:32.921481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caf6cc-37e2-4164-9c88-10e86a84588f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.921769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caf6cc-37e2-4164-9c88-10e86a84588f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.922040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caf6cc-37e2-4164-9c88-10e86a84588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.940136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a1caf6cc-37e2-4164-9c88-10e86a84588f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.941360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a1caf6cc-37e2-4164-9c88-10e86a84588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.949565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 806/1612] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.950876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:32.951341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:32.951468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:32.959094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:32.974779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:32.975984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:32.983800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 807/1613] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.984621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:32.994971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 807/1614] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:32.995929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.007054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a44027-a89a-4002-8b54-31fde948b1f9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.012196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.012901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.013186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.013479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.014104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.016030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.028629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:33.044534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72150-f31c-40f1-9951-f31d3ba71078 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.046569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.047219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.047490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.047758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.057022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.058434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.067588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 808/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:33.073346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.076505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868bfb6-30ee-4d42-be2a-6a359aca76b9 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.080321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.080890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.081161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.081514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.091500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.094496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.098020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 808/1616] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:32 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:33.098020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.103212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 809/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:33.104095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.114902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad1e1-3b5a-45c5-9402-d61b8d392fe5 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.157489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c0491-f755-4d67-81fb-bedb40b45986 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.161665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.162279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.162555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.162822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.175254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.176814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.176814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.176814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] PATH_INFO: `/v3/projects/9277ea301c0748748b55ad5e766d130b/users/7dac083b3ce6441da8a31bcbfe31b661/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.192566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9277ea301c0748748b55ad5e766d130b, user_id=7dac083b3ce6441da8a31bcbfe31b661, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.196117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.229222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.230710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.240809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 809/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:33.241700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.249974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5a98-e46d-4ee5-b9ef-ceb9309aaa7a service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.253798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.254389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.254669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.254936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.265756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.267549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.277358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 810/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:33.278224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.286792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c8fc-ff56-4459-9115-4f918b8becd0 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.293239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 810/1620] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:33 2022] PUT /identity/v3/projects/9277ea301c0748748b55ad5e766d130b/users/7dac083b3ce6441da8a31bcbfe31b661/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:33.293239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.354749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.355759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.356055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.356557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.361325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:33.367904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f761e-83c1-40cf-99c7-3ce33ad45483 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.373836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.374280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.374568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.374838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.375481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.377488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.421921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.423441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.434321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 811/1621] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:33.435195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.444302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15d628-9059-41be-9383-773cea2f09d3 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.449681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.450272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.450548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.450818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.451494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.453423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.502902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 811/1622] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:33 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:33.502902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.540598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:33.565635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a231c-d4db-48e5-918a-cac2b2978b2e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.570889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.571551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.571834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.572185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.572833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.574481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.585126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 812/1623] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:33.586460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.594465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 812/1624] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:33 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:33.595492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.597020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe13ef-2e3b-4f4d-aa05-ce1b69894eb6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.601872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c7aaa-fd56-4f5b-b732-f0e9b5c4c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.602563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c7aaa-fd56-4f5b-b732-f0e9b5c4c0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.604439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c7aaa-fd56-4f5b-b732-f0e9b5c4c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.604799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c7aaa-fd56-4f5b-b732-f0e9b5c4c0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.605510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c7aaa-fd56-4f5b-b732-f0e9b5c4c0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.607650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c7aaa-fd56-4f5b-b732-f0e9b5c4c0b9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.614591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcecf7-636b-4df6-8e23-795d2c9abd32 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.620059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.622476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.622476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.622476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] PATH_INFO: `/v3/projects/9277ea301c0748748b55ad5e766d130b/users/7dac083b3ce6441da8a31bcbfe31b661/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.623554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9277ea301c0748748b55ad5e766d130b, user_id=7dac083b3ce6441da8a31bcbfe31b661, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.626285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.644806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 813/1625] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:33 2022] PUT /identity/v3/projects/9277ea301c0748748b55ad5e766d130b/users/7dac083b3ce6441da8a31bcbfe31b661/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:33.645838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.716641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990162ce-7ad9-467d-803f-5482a4e7ccba admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.722360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 813/1626] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:33 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:33.722360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.722360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.722360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.722688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.723887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.724080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.724381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.725142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.726838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.742707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 814/1627] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:33.742707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.753608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8bd2f-7755-4ce4-ab13-56718c6a8a2e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.755973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] MFA Rules not processed for user `7dac083b3ce6441da8a31bcbfe31b661`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:33.758815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:33.759858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.760461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.760868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.761191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.761915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.763620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.780003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 815/1628] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:33.781140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.795670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efa392-d709-4983-acee-3b3ba3b380b3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.805568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.806641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.807245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.812657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] PATH_INFO: `/v3/projects/71cd2e4cb12b4b45b4c667129bb42054/users/da055e981dc34b1e927ef593b2111ad0/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.827209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] RBAC: Authorizing `identity:create_grant(project_id=71cd2e4cb12b4b45b4c667129bb42054, user_id=da055e981dc34b1e927ef593b2111ad0, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.829950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.846148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 814/1629] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:33.847096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.864308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dff27-dbfd-47dd-b240-02d107b9abcd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:33.866711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:33.867927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:33.868194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:33.868613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] PATH_INFO: `/v3/projects/7aff89a3d86641e78620f91ca1e5fa56/users/32a155db4699400ea4b550ac50c87f09/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:33.884198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aff89a3d86641e78620f91ca1e5fa56, user_id=32a155db4699400ea4b550ac50c87f09, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:33.891601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:33.946538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 816/1630] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:33 2022] PUT /identity/v3/projects/71cd2e4cb12b4b45b4c667129bb42054/users/da055e981dc34b1e927ef593b2111ad0/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:33.949090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:33.987548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:34.004592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 815/1631] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:33 2022] PUT /identity/v3/projects/7aff89a3d86641e78620f91ca1e5fa56/users/32a155db4699400ea4b550ac50c87f09/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:34.007757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.014084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246d692-e2e7-494e-846e-a7c85d6a206f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.082257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.083299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.084010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.084317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.118056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3015de-4066-4786-b129-485b416e0888 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.127577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.128347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.128629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.128904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.129923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.131384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.143565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 816/1632] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:34.147777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.159982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323b9a-0532-4451-980e-e26ecd49e0c0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.166694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.167659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.168024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.168460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.169193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.171047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.171810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.173369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.178554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:34.184186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 817/1633] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:34.185629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.188447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 817/1634] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:34.189871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.199873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06225a83-3078-45e1-875e-c2aa06819c41 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.205949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e6e14-eb54-4406-8dc2-837ee90cb27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.207179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e6e14-eb54-4406-8dc2-837ee90cb27e admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.207324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e6e14-eb54-4406-8dc2-837ee90cb27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.207645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e6e14-eb54-4406-8dc2-837ee90cb27e admin admin] PATH_INFO: `/v3/projects/71cd2e4cb12b4b45b4c667129bb42054/users/da055e981dc34b1e927ef593b2111ad0/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.210189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e6e14-eb54-4406-8dc2-837ee90cb27e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71cd2e4cb12b4b45b4c667129bb42054, user_id=da055e981dc34b1e927ef593b2111ad0, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.216994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15403aa8-20b3-433f-830e-a89f7c8bd278 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.217284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e6e14-eb54-4406-8dc2-837ee90cb27e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.226104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb0db0-bbb7-467f-b341-f037d9ac8a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.226830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0db0-bbb7-467f-b341-f037d9ac8a67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.227120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0db0-bbb7-467f-b341-f037d9ac8a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.227698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0db0-bbb7-467f-b341-f037d9ac8a67 admin admin] PATH_INFO: `/v3/projects/7aff89a3d86641e78620f91ca1e5fa56/users/32a155db4699400ea4b550ac50c87f09/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.230612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb0db0-bbb7-467f-b341-f037d9ac8a67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aff89a3d86641e78620f91ca1e5fa56, user_id=32a155db4699400ea4b550ac50c87f09, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.233776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb0db0-bbb7-467f-b341-f037d9ac8a67 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.244026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 818/1635] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:34 2022] PUT /identity/v3/projects/71cd2e4cb12b4b45b4c667129bb42054/users/da055e981dc34b1e927ef593b2111ad0/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:34.256096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.256706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.257702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.264642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 818/1636] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:34 2022] PUT /identity/v3/projects/7aff89a3d86641e78620f91ca1e5fa56/users/32a155db4699400ea4b550ac50c87f09/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:45:34.266075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.274899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.274899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.322527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] MFA Rules not processed for user `da055e981dc34b1e927ef593b2111ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:34.323920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:34.345373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] MFA Rules not processed for user `32a155db4699400ea4b550ac50c87f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:34.346622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:34.425489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 819/1637] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:34.441177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.461949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 819/1638] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:34.514246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cb659-11a0-46a6-b1f9-18cc5240cdc7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.521668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.523032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.523897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.524482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.524908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.531591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c099009-ac4a-40ed-8867-bb38af79cd0d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.535683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.536499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.536885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.537222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.547414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.548913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.553960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.556890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.567561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 820/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:45:34.569342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 820/1640] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:34.704444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:34.713801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e826f5-e68f-45e1-beca-10e30fc0e95a service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:34.728172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b083b6-d1a8-48a0-89e2-a13230223f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:34.728805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b083b6-d1a8-48a0-89e2-a13230223f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:34.729084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b083b6-d1a8-48a0-89e2-a13230223f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:34.729354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b083b6-d1a8-48a0-89e2-a13230223f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:34.811915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b083b6-d1a8-48a0-89e2-a13230223f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:34.813750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b083b6-d1a8-48a0-89e2-a13230223f32 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:34.836802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 821/1641] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:34.940782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:35.004785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:35.013786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cabe7-2c15-4854-8f86-a5ae2ff159fc service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:35.017739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:35.018355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:35.018631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:35.018899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:35.104051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:35.105828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:35.147352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 821/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:35.768382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539060-a253-47c0-9071-f36287568ef7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:35.768910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539060-a253-47c0-9071-f36287568ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:35.769006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539060-a253-47c0-9071-f36287568ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:35.811560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-22539060-a253-47c0-9071-f36287568ef7 None None] MFA Rules not processed for user `7f637cb3111a49c7b6c7d1514ab53719`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:35.813524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-22539060-a253-47c0-9071-f36287568ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:35.927669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 822/1643] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:35.933768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.048007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:36.052444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571533cb-0b93-4213-ae04-e2835bc42a3b service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.057886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.058499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.058845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.059114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.059773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.062240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.217017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 822/1644] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:35 2022] POST /identity/v3/projects => generated 405 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:36.219654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539060-a253-47c0-9071-f36287568ef7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.233026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539060-a253-47c0-9071-f36287568ef7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.294757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:36.331808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.332602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.332906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.333292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.333993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.337093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.453476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 823/1645] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:36 2022] POST /identity/v3/users => generated 309 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:36.460533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.472855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6a841-1555-4024-aead-a41253fac662 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.482191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.483099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.483834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.484217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.484574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.485425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.487116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.499495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f88bfa-904a-4de7-9e59-7c3be5df06fa admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.503643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 823/1646] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:36.507121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] PATH_INFO: `/v3/users/bbfb8ac1b9df4d968fde391a5ec6db4c` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbfb8ac1b9df4d968fde391a5ec6db4c)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: WARNING py.warnings [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: warnings.warn(msg) Sep 07 14:45:36.513990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: Sep 07 14:45:36.514422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.517536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.529301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7077353-57d7-4477-a941-12528ed4cab6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.541560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.542430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.542787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.543136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] PATH_INFO: `/v3/projects/370fb7dce2da481a8226c4fb8b68dd36/users/0708c70997a4424a85a9dc63e341e708/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.561282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370fb7dce2da481a8226c4fb8b68dd36, user_id=0708c70997a4424a85a9dc63e341e708, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.564112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.634950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 824/1647] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:36 2022] PUT /identity/v3/projects/370fb7dce2da481a8226c4fb8b68dd36/users/0708c70997a4424a85a9dc63e341e708/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:45:36.638240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.748357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a666af-82eb-46f0-a92a-40ef0464b239 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.772759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.772759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.772759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.772759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.773374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.773374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.782432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 825/1648] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:45:36.783424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:36.795645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f344a5c-52eb-408a-968f-3f40e93c215e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:36.801434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fa959-866b-4a19-9b55-0a20d00e1a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:36.802440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa959-866b-4a19-9b55-0a20d00e1a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:36.802790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa959-866b-4a19-9b55-0a20d00e1a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:36.803160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa959-866b-4a19-9b55-0a20d00e1a07 admin admin] PATH_INFO: `/v3/projects/370fb7dce2da481a8226c4fb8b68dd36/users/0708c70997a4424a85a9dc63e341e708/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:36.806939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fa959-866b-4a19-9b55-0a20d00e1a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370fb7dce2da481a8226c4fb8b68dd36, user_id=0708c70997a4424a85a9dc63e341e708, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:36.810439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fa959-866b-4a19-9b55-0a20d00e1a07 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:36.833301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 826/1649] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:36 2022] PUT /identity/v3/projects/370fb7dce2da481a8226c4fb8b68dd36/users/0708c70997a4424a85a9dc63e341e708/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:36.892591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbfb8ac1b9df4d968fde391a5ec6db4c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:36.957746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbfb8ac1b9df4d968fde391a5ec6db4c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:36.974497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbfb8ac1b9df4d968fde391a5ec6db4c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:36.975446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbfb8ac1b9df4d968fde391a5ec6db4c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:36.977092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbfb8ac1b9df4d968fde391a5ec6db4c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:37.011031 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 824/1650] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:36 2022] DELETE /identity/v3/users/bbfb8ac1b9df4d968fde391a5ec6db4c => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:37.015443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:37.015965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:37.016670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:37.042643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:37.044858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:37.124975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 827/1651] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:37.170548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:37.173243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:37.292710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3f0f-cd3d-438b-b198-15b099882608 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:37.298279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:37.299073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:37.299488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:37.299921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:37.312611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:37.314964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:37.333740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 825/1652] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:38.551166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:38.553560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:38.590386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c6a26-97b0-40dd-80b2-b00efcecabba None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:38.603809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:38.604834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:38.606665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:38.607171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] PATH_INFO: `/v3/projects/ee79ffed913244e18fec0abd00da8fa7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:38.614103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee79ffed913244e18fec0abd00da8fa7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:38.614103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 14:45:38.614103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Sep 07 14:45:38.614103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: Sep 07 14:45:38.614103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:39.252183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:39.284531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee79ffed913244e18fec0abd00da8fa7', 'request_id': 'req-4d7a4cb3-bb90-443b-86b4-b094e81b335d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:39.443261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee79ffed913244e18fec0abd00da8fa7', 'request_id': 'req-4d7a4cb3-bb90-443b-86b4-b094e81b335d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:39.459047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee79ffed913244e18fec0abd00da8fa7', 'request_id': 'req-4d7a4cb3-bb90-443b-86b4-b094e81b335d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:39.518107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] The token cache is being invalidate because project ee79ffed913244e18fec0abd00da8fa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:39.518512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:39.523756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:39.530396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 828/1653] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:38 2022] DELETE /identity/v3/projects/ee79ffed913244e18fec0abd00da8fa7 => generated 0 bytes in 976 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:39.532732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:39.548353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effadae-4a34-461a-a5ed-56e0be731207 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:39.750476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:39.786864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:39.799649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:39.799873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:39.812172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] PATH_INFO: `/v3/users/32a155db4699400ea4b550ac50c87f09` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:39.812172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a155db4699400ea4b550ac50c87f09)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:39.812172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:40.254793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a155db4699400ea4b550ac50c87f09'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:40.317188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a155db4699400ea4b550ac50c87f09'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:40.325965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a155db4699400ea4b550ac50c87f09'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:40.328662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a155db4699400ea4b550ac50c87f09'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:40.328988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-442c458a-32a8-4a20-b4ee-2775b156cfb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a155db4699400ea4b550ac50c87f09'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:40.392430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 826/1654] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:39 2022] DELETE /identity/v3/users/32a155db4699400ea4b550ac50c87f09 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:45:42.055237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:42.056622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:42.080276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a4cb3-bb90-443b-86b4-b094e81b335d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:42.224392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:42.225062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:42.225344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:42.225612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] PATH_INFO: `/v3/projects/7aff89a3d86641e78620f91ca1e5fa56` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:42.227190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aff89a3d86641e78620f91ca1e5fa56)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:42.229289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:42.430994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aff89a3d86641e78620f91ca1e5fa56', 'request_id': 'req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:42.513112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aff89a3d86641e78620f91ca1e5fa56', 'request_id': 'req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:42.522993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aff89a3d86641e78620f91ca1e5fa56', 'request_id': 'req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:42.542049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] The token cache is being invalidate because project 7aff89a3d86641e78620f91ca1e5fa56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:42.542608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:42.543540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-091fb7d4-798d-4a6a-ba3b-12f1c1501d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:42.546162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 829/1655] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:42 2022] DELETE /identity/v3/projects/7aff89a3d86641e78620f91ca1e5fa56 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:42.551528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afc6b3-c24e-4704-a156-e0b0da7cd2d9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:42.551528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afc6b3-c24e-4704-a156-e0b0da7cd2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:42.553294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afc6b3-c24e-4704-a156-e0b0da7cd2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:42.588613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-42afc6b3-c24e-4704-a156-e0b0da7cd2d9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:42.591285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-42afc6b3-c24e-4704-a156-e0b0da7cd2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:42.732433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:42.732714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:42.733089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:42.737565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 827/1656] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:42.757746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:42.758187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:42.758568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:42.798363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] MFA Rules not processed for user `0708c70997a4424a85a9dc63e341e708`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:42.801358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:42.806697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:42.806697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:42.823105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 828/1657] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:42.824757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:42.839436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab3c8-3b02-49d8-87db-27d02d7e7e3e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:42.846420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:42.848335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:42.848889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:42.849367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:42.850250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:42.850764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:42.854367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:42.874871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 829/1658] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:42.880272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:42.896631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:42.901667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc7568-4bac-4219-a4e3-3a2cf5df0d13 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:42.929476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:42.930458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:42.931021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:42.931613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:42.932516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:42.935959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:43.002586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 830/1659] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:43.058145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 830/1660] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:42 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:43.059142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:43.076566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177c464-51e6-4388-bf12-cebf3eaa16e9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:43.237165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:43.239061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:43.239061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:43.239061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:43.239445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:43.241557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:43.380469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 831/1661] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:43 2022] POST /identity/v3/users => generated 282 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:43.385858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:43.400233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28d726-f21b-486a-9ae3-7929903a5038 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:43.411164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:43.411528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:43.412609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:43.412891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:43.415684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:43.421303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:43.447819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 831/1662] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:43.448952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:43.468063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f99c7-6654-428e-aa4f-ca828b0ac893 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:43.488975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d14aa-9c16-4653-8cc5-e7d93c5dbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:43.489607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d14aa-9c16-4653-8cc5-e7d93c5dbafd admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:43.489883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d14aa-9c16-4653-8cc5-e7d93c5dbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:43.490150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d14aa-9c16-4653-8cc5-e7d93c5dbafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:43.490804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d14aa-9c16-4653-8cc5-e7d93c5dbafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:43.509525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d14aa-9c16-4653-8cc5-e7d93c5dbafd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:43.530956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 832/1663] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:43.535216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:43.547543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c3bc9-2793-4bc4-b04e-53cbe4cb1f2c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:43.554173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:43.555430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:43.555886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:43.556358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] PATH_INFO: `/v3/projects/b5447ca136224b579e686cbc438587f4/users/55e6e5c4b5c74d84aa40875bd6d1e7c4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:43.567749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:43.579008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5447ca136224b579e686cbc438587f4, user_id=55e6e5c4b5c74d84aa40875bd6d1e7c4, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:43.583021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:43.703327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 832/1664] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:43 2022] PUT /identity/v3/projects/b5447ca136224b579e686cbc438587f4/users/55e6e5c4b5c74d84aa40875bd6d1e7c4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:43.717475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31b3a-62b7-4601-8408-cc4a21c630f5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:43.717766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31b3a-62b7-4601-8408-cc4a21c630f5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:43.718048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31b3a-62b7-4601-8408-cc4a21c630f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:43.774381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-31b31b3a-62b7-4601-8408-cc4a21c630f5 None None] MFA Rules not processed for user `55e6e5c4b5c74d84aa40875bd6d1e7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:43.778297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-31b31b3a-62b7-4601-8408-cc4a21c630f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:43.918440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 833/1665] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:43.924589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:44.062657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289c17c-d9c0-4faf-bacd-b18ca4130a2f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:44.074000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:44.075065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.075923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.077326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.078320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:44.080802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:44.099237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fbe13-a3b6-455a-a380-8f17e7c8d344 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.100233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fbe13-a3b6-455a-a380-8f17e7c8d344 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.100669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fbe13-a3b6-455a-a380-8f17e7c8d344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.141658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c31fbe13-a3b6-455a-a380-8f17e7c8d344 None None] MFA Rules not processed for user `8eb5509df78d45acbebc332a889251dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:44.142910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c31fbe13-a3b6-455a-a380-8f17e7c8d344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:44.204675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 833/1666] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:43 2022] POST /identity/v3/projects => generated 365 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:44.211426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:44.245624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 834/1667] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:45:44.343523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20104ae4-25eb-4127-9752-524de212af85 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.344051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20104ae4-25eb-4127-9752-524de212af85 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.344536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20104ae4-25eb-4127-9752-524de212af85 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.346550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 835/1668] 10.222.0.57 () {54 vars in 962 bytes} [Wed Sep 7 14:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:45:44.371232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:44.380680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.381202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.381629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.412351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb5525-2ae6-49d1-a503-d4434d1eeca2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:44.428120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:44.428923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.429240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.429511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.430198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:44.433606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:44.472127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:44.473143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:44.563599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 834/1669] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:44 2022] POST /identity/v3/users => generated 289 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:45:44.564784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:44.598663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba163f44-cd28-4371-a4f3-cbbce66340ee admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:44.613156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:44.613896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.614386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.614666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.615436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:44.617924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:44.621245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 836/1670] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Sep 7 14:45:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:45:44.630279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:44.650334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 835/1671] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:44.658119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:44.683743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2b78a-a167-4492-acb5-b2787d63a73d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:44.717167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc6684-2979-4587-9dab-40bc7b855257 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:44.717266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:44.717266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.717266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.717266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.730889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:44.731787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:44.731787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:44.732056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] PATH_INFO: `/v3/projects/311093f91b7248c68465fd7ca3f9b69d/users/cafdb0fe684a4fb8895f323f2b1ac0ad/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:44.774936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311093f91b7248c68465fd7ca3f9b69d, user_id=cafdb0fe684a4fb8895f323f2b1ac0ad, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:44.777779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:44.882731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 836/1672] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:44 2022] PUT /identity/v3/projects/311093f91b7248c68465fd7ca3f9b69d/users/cafdb0fe684a4fb8895f323f2b1ac0ad/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:44.883838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:44.899700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:44.904622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:44.939046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 837/1673] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:44.980583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:45.031795 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba99304-0541-43ce-9662-7240cebaf4e6 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:45.044521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a04838-5bfc-4341-871c-03ec10f4d6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:45.045681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a04838-5bfc-4341-871c-03ec10f4d6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.046336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a04838-5bfc-4341-871c-03ec10f4d6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.047284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a04838-5bfc-4341-871c-03ec10f4d6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.048303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a04838-5bfc-4341-871c-03ec10f4d6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:45.050464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a04838-5bfc-4341-871c-03ec10f4d6c3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:45.079873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:45.080763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 837/1674] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:45.101629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232536c-6836-45e9-8392-0d59a6d095c2 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:45.121121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:45.121755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.122035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.122383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] PATH_INFO: `/v3/projects/311093f91b7248c68465fd7ca3f9b69d/users/cafdb0fe684a4fb8895f323f2b1ac0ad/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.129623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] RBAC: Authorizing `identity:create_grant(project_id=311093f91b7248c68465fd7ca3f9b69d, user_id=cafdb0fe684a4fb8895f323f2b1ac0ad, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:45.132391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:45.166627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 838/1675] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:45 2022] PUT /identity/v3/projects/311093f91b7248c68465fd7ca3f9b69d/users/cafdb0fe684a4fb8895f323f2b1ac0ad/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:45:45.178188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5390f-9fac-47aa-afed-32e969c621d5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.178188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5390f-9fac-47aa-afed-32e969c621d5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.178188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5390f-9fac-47aa-afed-32e969c621d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.239170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6fe5390f-9fac-47aa-afed-32e969c621d5 None None] MFA Rules not processed for user `cafdb0fe684a4fb8895f323f2b1ac0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:45.241748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6fe5390f-9fac-47aa-afed-32e969c621d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:45.373764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 838/1676] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:45.382927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:45.385447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:45.466906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:45.523010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd75a33-c043-4c1a-9cd9-90db6b80222c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:45.527735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee32a2-4db7-47aa-a97a-9e3078155cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:45.528559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee32a2-4db7-47aa-a97a-9e3078155cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.533923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee32a2-4db7-47aa-a97a-9e3078155cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.533923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee32a2-4db7-47aa-a97a-9e3078155cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.541157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee32a2-4db7-47aa-a97a-9e3078155cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:45.543041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee32a2-4db7-47aa-a97a-9e3078155cd8 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:45.556382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 839/1677] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:45.739083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2b98a-eb6a-45f1-8f31-397d9660d426 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.740133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2b98a-eb6a-45f1-8f31-397d9660d426 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.740639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2b98a-eb6a-45f1-8f31-397d9660d426 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.742737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 839/1678] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:45:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:45:45.799198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.800023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.801005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.830734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2b98a-eb6a-45f1-8f31-397d9660d426 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:45.848579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2b98a-eb6a-45f1-8f31-397d9660d426 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:45.879132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc329e-bdbd-4145-8cbb-2d32f9f3e234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:45.886719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc329e-bdbd-4145-8cbb-2d32f9f3e234 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:45.887759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc329e-bdbd-4145-8cbb-2d32f9f3e234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:45.889006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc329e-bdbd-4145-8cbb-2d32f9f3e234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:45.890306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:45.902150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:46.075368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc329e-bdbd-4145-8cbb-2d32f9f3e234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:46.077449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc329e-bdbd-4145-8cbb-2d32f9f3e234 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:46.091041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 840/1679] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:46.095832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 840/1680] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:45:46.183412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:46.186226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:46.230923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ed96-54bf-4e1f-b2ad-199efa82c700 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:46.239477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297697-c719-4277-96fb-c447b9680333 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:46.252406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297697-c719-4277-96fb-c447b9680333 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:46.256823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297697-c719-4277-96fb-c447b9680333 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:46.256823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297697-c719-4277-96fb-c447b9680333 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:46.266069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297697-c719-4277-96fb-c447b9680333 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:46.269510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297697-c719-4277-96fb-c447b9680333 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:46.284492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 841/1681] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:48.888636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e83778-1008-42fb-be7a-4b61f3357077 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:48.889180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e83778-1008-42fb-be7a-4b61f3357077 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:48.889275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e83778-1008-42fb-be7a-4b61f3357077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:48.988197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-c5e83778-1008-42fb-be7a-4b61f3357077 None None] MFA Rules not processed for user `eeab924bf7a646779b48f57f14710e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:48.995141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c5e83778-1008-42fb-be7a-4b61f3357077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:49.121526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:49.216237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 841/1682] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:49.271933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee67802-13f6-48f8-931f-cc2278a3cb78 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:49.272545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee67802-13f6-48f8-931f-cc2278a3cb78 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:49.272545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee67802-13f6-48f8-931f-cc2278a3cb78 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:49.274173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 842/1683] 10.222.0.57 () {54 vars in 962 bytes} [Wed Sep 7 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:45:49.286657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:49.287259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:49.287690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:49.349449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:49.351196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:49.478480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 842/1684] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Sep 7 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:49.480076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee67802-13f6-48f8-931f-cc2278a3cb78 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:49.507892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee67802-13f6-48f8-931f-cc2278a3cb78 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:49.518384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b256b-0a25-4418-a6a8-b79951890fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:49.519330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b256b-0a25-4418-a6a8-b79951890fc5 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:49.519901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b256b-0a25-4418-a6a8-b79951890fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:49.527825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b256b-0a25-4418-a6a8-b79951890fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:49.561370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b256b-0a25-4418-a6a8-b79951890fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:49.562995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b256b-0a25-4418-a6a8-b79951890fc5 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:49.607391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 843/1685] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:45:49.983918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:49.985408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:50.203118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8be297-4f56-4348-97ee-d1fc2ea5b7e4 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:50.214864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:50.216044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:50.216568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:50.217053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] PATH_INFO: `/v3/users/7f637cb3111a49c7b6c7d1514ab53719` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:50.218913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f637cb3111a49c7b6c7d1514ab53719)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:50.222186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:50.470695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae72b5-1c8f-469f-a35d-152694a261c3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:50.471601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae72b5-1c8f-469f-a35d-152694a261c3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:50.472073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae72b5-1c8f-469f-a35d-152694a261c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:50.496083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f637cb3111a49c7b6c7d1514ab53719'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:50.516048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f3ae72b5-1c8f-469f-a35d-152694a261c3 None None] MFA Rules not processed for user `9f46e69140e347669da323ad08edd7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:50.518010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f3ae72b5-1c8f-469f-a35d-152694a261c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:50.580868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f637cb3111a49c7b6c7d1514ab53719'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:50.597190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f637cb3111a49c7b6c7d1514ab53719'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:50.598568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f637cb3111a49c7b6c7d1514ab53719'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:50.600189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f637cb3111a49c7b6c7d1514ab53719'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:50.610923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:50.619417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 844/1686] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:50.630674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219ec3a-062f-43df-a915-d2c5dc51f35c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:50.631703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219ec3a-062f-43df-a915-d2c5dc51f35c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:50.634377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219ec3a-062f-43df-a915-d2c5dc51f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:50.639128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 843/1687] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:49 2022] DELETE /identity/v3/users/7f637cb3111a49c7b6c7d1514ab53719 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:50.672604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0219ec3a-062f-43df-a915-d2c5dc51f35c None None] MFA Rules not processed for user `9f46e69140e347669da323ad08edd7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:50.674304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0219ec3a-062f-43df-a915-d2c5dc51f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:50.711561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:50.768190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 845/1688] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:50.776887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:50.779627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:50.907604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23bd53-3405-436c-a0a4-c45f4eafa5f5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:50.921852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:50.922487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:50.922769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:50.923040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:50.927633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:50.947925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:50.949597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919594a-93da-4461-8eed-cd9674ab2011 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:51.012217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 844/1689] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:51.142358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b0006-c131-4348-8cf7-97c862b0a189 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:51.142671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b0006-c131-4348-8cf7-97c862b0a189 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:51.142946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b0006-c131-4348-8cf7-97c862b0a189 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:51.144623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 846/1690] 10.222.0.57 () {54 vars in 912 bytes} [Wed Sep 7 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:45:51.151035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:51.151552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:51.152008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:51.195868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:51.197121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:51.215214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 845/1691] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:51.228594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b0006-c131-4348-8cf7-97c862b0a189 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:51.231104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e53b0006-c131-4348-8cf7-97c862b0a189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:51.431485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b0006-c131-4348-8cf7-97c862b0a189 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:51.435494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:51.436127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:51.436454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:51.436725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:51.465986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:51.467447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:51.530311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 847/1692] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:51.544728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:51.551188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:51.736061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244755a-55ad-47a2-9044-3a17c2ee230f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:51.742632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:51.743280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:51.744218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:51.745426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] PATH_INFO: `/v3/projects/22099e81fa2b4333ba3d9806ae2c4b5b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:51.748312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22099e81fa2b4333ba3d9806ae2c4b5b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:51.750677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:51.781494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:51.783208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:51.808541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe31ce0-c073-4b4e-8d5b-be9c5d93b884 service glance] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:51.822157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:51.834564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:51.834967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:51.840006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] PATH_INFO: `/v3/users/55e6e5c4b5c74d84aa40875bd6d1e7c4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:51.843439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e6e5c4b5c74d84aa40875bd6d1e7c4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:51.846305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:52.007706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22099e81fa2b4333ba3d9806ae2c4b5b', 'request_id': 'req-7961fbba-c9a7-4681-a92e-18eb629e599a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.088463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22099e81fa2b4333ba3d9806ae2c4b5b', 'request_id': 'req-7961fbba-c9a7-4681-a92e-18eb629e599a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.099228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22099e81fa2b4333ba3d9806ae2c4b5b', 'request_id': 'req-7961fbba-c9a7-4681-a92e-18eb629e599a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.141186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e6e5c4b5c74d84aa40875bd6d1e7c4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.141668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] The token cache is being invalidate because project 22099e81fa2b4333ba3d9806ae2c4b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:52.142347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.142805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e6e5c4b5c74d84aa40875bd6d1e7c4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.144043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e6e5c4b5c74d84aa40875bd6d1e7c4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.144688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7961fbba-c9a7-4681-a92e-18eb629e599a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.148314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 846/1693] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:51 2022] DELETE /identity/v3/projects/22099e81fa2b4333ba3d9806ae2c4b5b => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:52.158015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.158299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.158572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.213463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] MFA Rules not processed for user `8eb5509df78d45acbebc332a889251dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:52.214731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:52.268532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e6e5c4b5c74d84aa40875bd6d1e7c4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.287398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 847/1694] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:52.289614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:52.290999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:52.296903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a1e24a3f-8f4d-4238-9b09-ed43bc1baf8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e6e5c4b5c74d84aa40875bd6d1e7c4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.341801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 848/1695] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:51 2022] DELETE /identity/v3/users/55e6e5c4b5c74d84aa40875bd6d1e7c4 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:52.346581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e25f91-b88b-4035-a986-f7e72badd45c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.348059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e25f91-b88b-4035-a986-f7e72badd45c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.348617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e25f91-b88b-4035-a986-f7e72badd45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.367673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63442228-8b98-4620-8982-aa50d6fc3939 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:52.375076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-29e25f91-b88b-4035-a986-f7e72badd45c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:52.376696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-29e25f91-b88b-4035-a986-f7e72badd45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:52.418525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:52.446743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:52.447678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.448099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.448557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] PATH_INFO: `/v3/users/0708c70997a4424a85a9dc63e341e708` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.449929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0708c70997a4424a85a9dc63e341e708)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:52.452119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:52.487764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 849/1696] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:52.489946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13e684-151e-442d-a02b-ad9e1d8104a4 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.490238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13e684-151e-442d-a02b-ad9e1d8104a4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.490508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13e684-151e-442d-a02b-ad9e1d8104a4 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.492941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 850/1697] 10.222.0.57 () {54 vars in 912 bytes} [Wed Sep 7 14:45:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:45:52.502652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.502652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.502652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.540471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:52.543796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:52.552186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:52.669012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 851/1698] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:52.673363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:52.683050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:52.697295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0708c70997a4424a85a9dc63e341e708'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.796494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac85e3-9ba3-4488-8609-df8cdeb4423a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:52.802411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:52.803120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.803460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.803929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.817100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:52.817100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:52.829805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 852/1699] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:52.830758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:52.832732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:52.865747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0708c70997a4424a85a9dc63e341e708'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.876449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0708c70997a4424a85a9dc63e341e708'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.884649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0708c70997a4424a85a9dc63e341e708'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.888009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8f3566b0-4e94-40bc-9ea2-40519ed1e165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0708c70997a4424a85a9dc63e341e708'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:52.935149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 848/1700] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:52 2022] DELETE /identity/v3/users/0708c70997a4424a85a9dc63e341e708 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:52.963739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e9635-2780-4a2e-9930-e04396b240b1 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.964033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e9635-2780-4a2e-9930-e04396b240b1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.965854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e9635-2780-4a2e-9930-e04396b240b1 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:52.984748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 849/1701] 10.222.0.57 () {54 vars in 912 bytes} [Wed Sep 7 14:45:52 2022] GET /identity => generated 270 bytes in 49 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:45:52.987760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f94a4-10f8-4361-9194-c98cc6d0368e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:52.988046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f94a4-10f8-4361-9194-c98cc6d0368e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:52.988446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f94a4-10f8-4361-9194-c98cc6d0368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.041146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e91618-8f4e-4bf0-a9dd-81fa46895161 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:53.108998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e27f94a4-10f8-4361-9194-c98cc6d0368e None None] MFA Rules not processed for user `9d8116d648d64a62af52d9db6a607e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:53.110984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e27f94a4-10f8-4361-9194-c98cc6d0368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.171485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:53.172813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:53.173707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:53.174148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.177522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.279453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 850/1702] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:53.282482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c880e8-b0dc-4690-940d-1a73b88c8654 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:53.283291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c880e8-b0dc-4690-940d-1a73b88c8654 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:53.283790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c880e8-b0dc-4690-940d-1a73b88c8654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.378344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-62c880e8-b0dc-4690-940d-1a73b88c8654 None None] MFA Rules not processed for user `b50c8a35e28e44bba3a4b4e56057a3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:53.382598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-62c880e8-b0dc-4690-940d-1a73b88c8654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.409747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:53.418818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:53.451150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 853/1703] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:53.455667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:53.461254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.619116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f3801-bb9a-4def-804c-0342b699a11b service glance] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:53.620618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 851/1704] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:53.627080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:53.628033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:53.628199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:53.628414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:53.628798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:53.629048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.629273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] PATH_INFO: `/v3/users/d261369de0bf488ba4645da90e41ade3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.632501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d261369de0bf488ba4645da90e41ade3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:53.638090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:53.675678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] MFA Rules not processed for user `9d8116d648d64a62af52d9db6a607e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:53.677775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.705921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 852/1705] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:53.718415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:53.720353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.742583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278883d-1037-4f41-a2be-bb26cc61b08b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:53.752702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:53.754190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:53.760726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:53.764694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.781425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:53.782924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:53.796447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 853/1706] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:45:53.797332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:53.801396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:53.827330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d207-a007-410b-9fc6-b688660d6abb service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:53.832073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:53.832879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:53.833150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:53.833423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] PATH_INFO: `/v3/projects/370fb7dce2da481a8226c4fb8b68dd36` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:53.834648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370fb7dce2da481a8226c4fb8b68dd36)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:53.836764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:53.922948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd261369de0bf488ba4645da90e41ade3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:53.924521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd261369de0bf488ba4645da90e41ade3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:53.925173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd261369de0bf488ba4645da90e41ade3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:53.929373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:53.989284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370fb7dce2da481a8226c4fb8b68dd36', 'request_id': 'req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.026569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd261369de0bf488ba4645da90e41ade3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.036941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd261369de0bf488ba4645da90e41ade3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.092806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:54.104738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 854/1707] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:53 2022] DELETE /identity/v3/users/d261369de0bf488ba4645da90e41ade3 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:54.105726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:54.108586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.111658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370fb7dce2da481a8226c4fb8b68dd36', 'request_id': 'req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.121279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370fb7dce2da481a8226c4fb8b68dd36', 'request_id': 'req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.146235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] The token cache is being invalidate because project 370fb7dce2da481a8226c4fb8b68dd36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:54.147735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.169349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-53b28d6a-b477-47eb-ab58-c96f8f04e1e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.169349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 854/1708] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:53 2022] DELETE /identity/v3/projects/370fb7dce2da481a8226c4fb8b68dd36 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:54.169349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268ac8a-ec20-4cc3-99a5-4d5d6dd46ebf None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.169349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268ac8a-ec20-4cc3-99a5-4d5d6dd46ebf None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.169349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268ac8a-ec20-4cc3-99a5-4d5d6dd46ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.227735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a268ac8a-ec20-4cc3-99a5-4d5d6dd46ebf None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:54.230174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a268ac8a-ec20-4cc3-99a5-4d5d6dd46ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.297181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6633ecd-8cf8-446d-b54f-78cf66376fd0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:54.308443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:54.309099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.309375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.309646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] PATH_INFO: `/v3/projects/b5447ca136224b579e686cbc438587f4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.311310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5447ca136224b579e686cbc438587f4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:54.317393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:54.342230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 855/1709] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:54.343558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4efea8-1c26-4aa0-b144-d677862683f7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.343845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4efea8-1c26-4aa0-b144-d677862683f7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.344198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4efea8-1c26-4aa0-b144-d677862683f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.383081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-eb4efea8-1c26-4aa0-b144-d677862683f7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:54.384424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-eb4efea8-1c26-4aa0-b144-d677862683f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.394965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 856/1710] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:54.396427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e95a9-21d0-4a6c-8092-e99427b6e915 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.396862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e95a9-21d0-4a6c-8092-e99427b6e915 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.397271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e95a9-21d0-4a6c-8092-e99427b6e915 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.405576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 857/1711] 10.222.0.57 () {54 vars in 912 bytes} [Wed Sep 7 14:45:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:45:54.406685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e95a9-21d0-4a6c-8092-e99427b6e915 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:54.408436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-627e95a9-21d0-4a6c-8092-e99427b6e915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.461673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:54.469524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5447ca136224b579e686cbc438587f4', 'request_id': 'req-2b0f218f-5b85-440c-89ca-467d8e996eb0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.497151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e95a9-21d0-4a6c-8092-e99427b6e915 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:54.534819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5447ca136224b579e686cbc438587f4', 'request_id': 'req-2b0f218f-5b85-440c-89ca-467d8e996eb0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.545732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5447ca136224b579e686cbc438587f4', 'request_id': 'req-2b0f218f-5b85-440c-89ca-467d8e996eb0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.564938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] The token cache is being invalidate because project b5447ca136224b579e686cbc438587f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:54.565495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.566297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2b0f218f-5b85-440c-89ca-467d8e996eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:54.572225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 855/1712] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:54 2022] DELETE /identity/v3/projects/b5447ca136224b579e686cbc438587f4 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:54.572225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f97d12-8331-4844-8cb3-05f8e131ddc1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.572225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f97d12-8331-4844-8cb3-05f8e131ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.572225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f97d12-8331-4844-8cb3-05f8e131ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.592394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:54.593999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-65f97d12-8331-4844-8cb3-05f8e131ddc1 None None] MFA Rules not processed for user `eeab924bf7a646779b48f57f14710e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:54.594587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.595224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.595363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-65f97d12-8331-4844-8cb3-05f8e131ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.595904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.598230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.696431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 856/1713] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:54.698303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bce6e-5f46-4716-945b-6f876ae62726 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.698764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bce6e-5f46-4716-945b-6f876ae62726 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.699199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bce6e-5f46-4716-945b-6f876ae62726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.728901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:54.731547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:54.733494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-214bce6e-5f46-4716-945b-6f876ae62726 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:54.734903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-214bce6e-5f46-4716-945b-6f876ae62726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.747178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 857/1714] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:54.750498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 858/1715] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:54.750628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.750712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:54.750795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.750795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.750920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.769034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb800617-72e2-4c22-9b43-cfd5f5b9b4e3 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:54.802312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:54.803449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.804472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.805657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] PATH_INFO: `/v3/users/cafdb0fe684a4fb8895f323f2b1ac0ad` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.808428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] MFA Rules not processed for user `b50c8a35e28e44bba3a4b4e56057a3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:54.809677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.810024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafdb0fe684a4fb8895f323f2b1ac0ad)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:54.812665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:54.835374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 858/1716] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:54.844625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:54.848820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:54.941985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c24e1-972e-40d2-ab8c-f65aecc8b69b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:54.949642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:54.950719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:54.950719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:54.950719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:54.953099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:55.031754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:55.034454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:55.045464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 859/1717] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:55.046587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:55.060020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e71d2b-5951-4956-8dcc-b6a68afeb4b4 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:55.069952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:55.072067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:55.073361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:55.075373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:55.076450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:55.076883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:55.080932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:55.112812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 860/1718] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:55.116717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:55.119893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:55.134453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafdb0fe684a4fb8895f323f2b1ac0ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:55.244847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafdb0fe684a4fb8895f323f2b1ac0ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:55.254660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafdb0fe684a4fb8895f323f2b1ac0ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:55.255598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafdb0fe684a4fb8895f323f2b1ac0ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:55.256439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafdb0fe684a4fb8895f323f2b1ac0ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:55.263275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce22d8-51f4-42c8-a090-2f40d093b32a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:55.270619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:55.271256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:55.271538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:55.271908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:55.274353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:55.293966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:55.293966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:55.305468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 861/1719] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:55.306493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:55.318923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:55.321972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e35d0-067a-4ddd-a22a-9507a1f88c1b service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:55.322615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 859/1720] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:54 2022] DELETE /identity/v3/users/cafdb0fe684a4fb8895f323f2b1ac0ad => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:55.326617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:55.337445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:55.394894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:55.396414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:55.396808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:55.397184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:55.400402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:55.416866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:55.419071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:55.429665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 862/1721] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:45:55.482154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf562-d308-4741-9126-a96075248962 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:55.494082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:55.496622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:55.496911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:55.497180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:55.497831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:55.503213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:55.700601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 860/1722] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:55 2022] POST /identity/v3/projects => generated 387 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:55.700952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:55.720169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f608b4-902a-46f7-bc65-9f92188ac2e0 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:55.820913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:55.821736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:55.822187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:55.822600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:55.823461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:55.829898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:56.003222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 863/1723] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:55 2022] POST /identity/v3/users => generated 294 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:56.006532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:56.018571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce62d3-95de-46ce-b5b6-bdfea4471db1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:56.024753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:56.034477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:56.034477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:56.034477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:56.034477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:56.034477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:56.047300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:56.047987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 861/1724] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:56.050086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:56.059649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:56.083565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056b9ce-7a7c-4f63-979c-e714f0d98045 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:56.104187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:56.104589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5f95-e4fa-4317-876f-01d3b1ae87eb admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:56.106092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:56.107339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:56.108187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] PATH_INFO: `/v3/projects/f669e473264f4f52b07819cc9ca21576` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:56.112868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f669e473264f4f52b07819cc9ca21576)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:56.119350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:56.125278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:56.125278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:56.125278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:56.125278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:56.127811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:56.149370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:56.164160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 862/1725] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:56.169253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:56.195411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e51f0b-c612-4812-9a30-f3966e820a92 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:56.206643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a45680-5bef-4fcd-84aa-926c795f2244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:56.207391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a45680-5bef-4fcd-84aa-926c795f2244 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:56.207674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a45680-5bef-4fcd-84aa-926c795f2244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:56.207948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a45680-5bef-4fcd-84aa-926c795f2244 admin admin] PATH_INFO: `/v3/projects/ff801b76186c43d1a95720a2eec820f8/users/0a3a3ad93030477dbf23ea33482be13b/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:56.275967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a45680-5bef-4fcd-84aa-926c795f2244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff801b76186c43d1a95720a2eec820f8, user_id=0a3a3ad93030477dbf23ea33482be13b, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:56.279347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a45680-5bef-4fcd-84aa-926c795f2244 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:56.374624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 863/1726] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:56 2022] PUT /identity/v3/projects/ff801b76186c43d1a95720a2eec820f8/users/0a3a3ad93030477dbf23ea33482be13b/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:56.380334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:56.380616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:56.380886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:56.383290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f669e473264f4f52b07819cc9ca21576', 'request_id': 'req-07b55912-b7ab-40fd-9b97-8b162b899f10'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:56.437458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] MFA Rules not processed for user `0a3a3ad93030477dbf23ea33482be13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:56.439435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:56.477628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f669e473264f4f52b07819cc9ca21576', 'request_id': 'req-07b55912-b7ab-40fd-9b97-8b162b899f10'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:56.489501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f669e473264f4f52b07819cc9ca21576', 'request_id': 'req-07b55912-b7ab-40fd-9b97-8b162b899f10'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:56.535336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] The token cache is being invalidate because project f669e473264f4f52b07819cc9ca21576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:56.535336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:56.535336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:56.539604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 864/1727] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:56 2022] DELETE /identity/v3/projects/f669e473264f4f52b07819cc9ca21576 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:56.541575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:56.546919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:56.557307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:56.629417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 864/1728] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:45:56.632630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:56.634786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:56.764880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b55912-b7ab-40fd-9b97-8b162b899f10 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:56.830053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43a1b7-8ed8-4b8f-af51-fbfe833e0910 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:56.869088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:56.870398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:56.870698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:56.872099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] PATH_INFO: `/v3/projects/311093f91b7248c68465fd7ca3f9b69d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:56.876627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=311093f91b7248c68465fd7ca3f9b69d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:56.885118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:57.017553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:57.017553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:57.018197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:57.018491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] PATH_INFO: `/v3/users/da055e981dc34b1e927ef593b2111ad0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:57.020328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da055e981dc34b1e927ef593b2111ad0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:57.026808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:57.185473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311093f91b7248c68465fd7ca3f9b69d', 'request_id': 'req-3c14a033-de83-4a44-94a1-cfe03cebd1fd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.298935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311093f91b7248c68465fd7ca3f9b69d', 'request_id': 'req-3c14a033-de83-4a44-94a1-cfe03cebd1fd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.308652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311093f91b7248c68465fd7ca3f9b69d', 'request_id': 'req-3c14a033-de83-4a44-94a1-cfe03cebd1fd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.324558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da055e981dc34b1e927ef593b2111ad0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.324776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da055e981dc34b1e927ef593b2111ad0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.325585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da055e981dc34b1e927ef593b2111ad0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.359118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] The token cache is being invalidate because project 311093f91b7248c68465fd7ca3f9b69d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:57.359118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.367934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.377236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 865/1729] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:56 2022] DELETE /identity/v3/projects/311093f91b7248c68465fd7ca3f9b69d => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:57.380393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:57.382353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:57.388490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:57.449490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da055e981dc34b1e927ef593b2111ad0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.462726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8db79f4-b2b1-4a52-8d67-7e31d16ba927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da055e981dc34b1e927ef593b2111ad0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:57.524693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 865/1730] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:45:56 2022] DELETE /identity/v3/users/da055e981dc34b1e927ef593b2111ad0 => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:45:57.526066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe17af-d875-42c8-a8c1-1ec701ff4e38 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:57.526359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe17af-d875-42c8-a8c1-1ec701ff4e38 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:57.526672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe17af-d875-42c8-a8c1-1ec701ff4e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:57.543020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14a033-de83-4a44-94a1-cfe03cebd1fd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:57.596544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4ffe17af-d875-42c8-a8c1-1ec701ff4e38 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:57.597910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4ffe17af-d875-42c8-a8c1-1ec701ff4e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:57.658467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:57.660909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:57.661221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:57.661959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:57.663212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:57.665846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:57.753741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 866/1731] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:57.760203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e57ba-1a1e-4775-ab9b-4687fda0e417 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:57.760718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e57ba-1a1e-4775-ab9b-4687fda0e417 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:57.761140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e57ba-1a1e-4775-ab9b-4687fda0e417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:57.789660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-369e57ba-1a1e-4775-ab9b-4687fda0e417 None None] MFA Rules not processed for user `9d8116d648d64a62af52d9db6a607e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:57.791715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-369e57ba-1a1e-4775-ab9b-4687fda0e417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:57.868979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 867/1732] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:57.870655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:57.870947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:57.871258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:57.895395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:57.896761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:57.908287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 868/1733] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:57.912116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:57.915774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 866/1734] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:57 2022] POST /identity/v3/projects => generated 385 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:57.916768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:57.923379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f342044-1353-4c0d-95fc-47d40d7c80ec None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.072618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.077478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.077766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.078771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.079279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:58.079583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.082453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.100920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c59d0-cd56-428a-a87b-6013d3bc39d2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.106365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.106983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.107261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.107533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.108326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.110607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.119243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 869/1735] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:58.120480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:58.141015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02062ec0-99f0-4394-a3f8-6e8a9903d92f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.149310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.149531 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.149805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.150081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.150822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.152965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.247420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 867/1736] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:57 2022] POST /identity/v3/users => generated 299 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:58.253901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:58.267650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493014ef-4aef-4f3d-8b32-48a3d8597f3b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.274366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.278775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.278775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.278775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.278775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.280253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.299322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 868/1737] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:58.304393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:58.307608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 870/1738] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:45:58 2022] POST /identity/v3/projects => generated 367 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:45:58.311338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:58.328664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdf827-9b2f-46ea-9551-2568d6e2b7f9 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.424109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8dd47-06ca-4835-86d1-74e7440233eb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.430912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.431593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.431878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.432172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.434317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.436597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.550932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.555396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.560568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.564971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] PATH_INFO: `/v3/projects/fc824547a9684d11924f2f30233caa10/users/f19b283699f040a3acbea0bc709eabf5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.600976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 871/1739] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:45:58 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:45:58.602191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:58.603263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc824547a9684d11924f2f30233caa10, user_id=f19b283699f040a3acbea0bc709eabf5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.606520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:58.608109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.636853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef4ec0-8373-4905-abf3-8cb9d6f9c2e2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.642492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:58.643206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:58.643554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:58.643899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] PATH_INFO: `/v3/projects/71cd2e4cb12b4b45b4c667129bb42054` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:58.645283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] RBAC: Authorizing `identity:delete_project(project_id=71cd2e4cb12b4b45b4c667129bb42054)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:58.647491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:58.755100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 869/1740] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:58 2022] PUT /identity/v3/projects/fc824547a9684d11924f2f30233caa10/users/f19b283699f040a3acbea0bc709eabf5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:58.755100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:58.836065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71cd2e4cb12b4b45b4c667129bb42054', 'request_id': 'req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:58.871265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:45:58.928752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71cd2e4cb12b4b45b4c667129bb42054', 'request_id': 'req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:58.943158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71cd2e4cb12b4b45b4c667129bb42054', 'request_id': 'req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:58.957070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0770fc7-9bd0-4af9-ad0f-fe1ebc808866 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:58.994611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] The token cache is being invalidate because project 71cd2e4cb12b4b45b4c667129bb42054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:45:58.994915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:59.002252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:45:59.012943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 872/1741] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:45:58 2022] DELETE /identity/v3/projects/71cd2e4cb12b4b45b4c667129bb42054 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:59.015523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:59.019824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:59.099295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1261-6ec5-4e06-b816-b90b7cc655e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:59.100416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1261-6ec5-4e06-b816-b90b7cc655e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.100763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1261-6ec5-4e06-b816-b90b7cc655e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.101035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1261-6ec5-4e06-b816-b90b7cc655e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.102343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc1261-6ec5-4e06-b816-b90b7cc655e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:59.103327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc1261-6ec5-4e06-b816-b90b7cc655e2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:59.141918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 870/1742] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:59.144293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.144793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.145296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.177839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:59.182756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:59.204005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 871/1743] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:59.212135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:59.213978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:59.226642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d9a3f-4de9-4b63-a4dc-85bac165311b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:59.227177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee0d6e-17ef-46ee-8778-77d49feb16f2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:59.233525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fadef-172a-4708-a422-5cf7cab023ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:59.234215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fadef-172a-4708-a422-5cf7cab023ee admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.234497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fadef-172a-4708-a422-5cf7cab023ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.234825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fadef-172a-4708-a422-5cf7cab023ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.236601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fadef-172a-4708-a422-5cf7cab023ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:59.237673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fadef-172a-4708-a422-5cf7cab023ee admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:59.248328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:59.253129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.253129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.253129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.253129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:59.255894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:59.265063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 872/1744] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:59.267096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.267454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.267845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.282457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 873/1745] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:45:59.292501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:59.296064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:59.297685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:59.312515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c6ecd-6546-46fd-9094-281ffd63b9d5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:59.320745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 873/1746] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:59.336497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cc4ce-f580-47a6-83ab-51c7a4a5adf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:59.336750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:59.337321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cc4ce-f580-47a6-83ab-51c7a4a5adf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.338444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cc4ce-f580-47a6-83ab-51c7a4a5adf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.338800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cc4ce-f580-47a6-83ab-51c7a4a5adf7 admin admin] PATH_INFO: `/v3/projects/2d3e14e5745e40b4ab848e0a2bb241bc/users/3036bb7a137741d7ac67821d0e7919ca/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.362548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cee54-a149-4924-abdc-b7ac113a833c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:59.372484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:59.377118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.378573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.378973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] PATH_INFO: `/v3/projects/fc824547a9684d11924f2f30233caa10/users/f19b283699f040a3acbea0bc709eabf5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.379655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cc4ce-f580-47a6-83ab-51c7a4a5adf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d3e14e5745e40b4ab848e0a2bb241bc, user_id=3036bb7a137741d7ac67821d0e7919ca, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:59.384235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc824547a9684d11924f2f30233caa10, user_id=f19b283699f040a3acbea0bc709eabf5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:59.388365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cc4ce-f580-47a6-83ab-51c7a4a5adf7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:59.388748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:59.416595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 874/1747] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:59 2022] PUT /identity/v3/projects/fc824547a9684d11924f2f30233caa10/users/f19b283699f040a3acbea0bc709eabf5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:59.418883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:59.470286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 874/1748] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:45:59 2022] PUT /identity/v3/projects/2d3e14e5745e40b4ab848e0a2bb241bc/users/3036bb7a137741d7ac67821d0e7919ca/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:45:59.470286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a4cd-a967-4f77-ab60-04725b5783fa None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.470286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a4cd-a967-4f77-ab60-04725b5783fa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.470286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a4cd-a967-4f77-ab60-04725b5783fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.558394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6262a4cd-a967-4f77-ab60-04725b5783fa None None] MFA Rules not processed for user `f19b283699f040a3acbea0bc709eabf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:59.558394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6262a4cd-a967-4f77-ab60-04725b5783fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:59.561175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7441c-54f4-4ce1-b670-a84ea672af6c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:45:59.716026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 875/1749] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:45:59.717492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.718061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.718061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.779822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] MFA Rules not processed for user `3036bb7a137741d7ac67821d0e7919ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:45:59.781643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:45:59.798071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:45:59.799409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:45:59.799822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:45:59.800351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:45:59.804607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:45:59.804607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:45:59.805517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:45:59.840912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 875/1750] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:45:59.840912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:45:59.851309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:00.019027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 876/1751] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:00.019737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.049395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320a5d8-ed86-4703-8ac4-1080376e8cb7 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.059388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.061048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4973e81-51cd-43e5-b3a4-795963ca8b2e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.061825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.061934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.062820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.073740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.074194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.074551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.075419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.076277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.080304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.081614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.087440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.100749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 876/1752] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:45:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:00.101704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.112396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9fb8b-4798-42bc-8b4e-e1112773a39d service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.117506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.118263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.118608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.118943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.119689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.122040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.226255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 877/1753] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:00 2022] POST /identity/v3/projects => generated 355 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:00.227200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.285136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 877/1754] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:00 2022] POST /identity/v3/projects => generated 369 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:00.286561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.297179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f25076-5778-4484-aa49-8ea461b56e6f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.397106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5bdb2-edc8-42c7-a714-072768e0c18c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.407882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.408558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.409381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.409381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.409721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.411669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.426164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.427245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.428290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.429129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] PATH_INFO: `/v3/users/0a3a3ad93030477dbf23ea33482be13b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.431653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3a3ad93030477dbf23ea33482be13b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.433267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.533450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 878/1755] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:00 2022] POST /identity/v3/users => generated 278 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:00.534488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.550972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bb14-c80e-4781-862a-f2dc4e6c5fa1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.560050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.561052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.561593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.561872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.562791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.566525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.697761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3a3ad93030477dbf23ea33482be13b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:00.704203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 879/1756] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:00 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:00.704846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.710309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3a3ad93030477dbf23ea33482be13b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:00.710519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:00.711403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3a3ad93030477dbf23ea33482be13b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:00.783528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacd8a4-656f-4475-ade3-228d92b7dbbf admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.795303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:00.798134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.799545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.800212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.800729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.801593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.803931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.818656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 880/1757] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:00.826953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.826953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:00.844650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c9ab2-e2e0-4a9d-befc-9448059325e3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.880323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:00.880935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3a3ad93030477dbf23ea33482be13b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:00.881371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:00.882385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:00.882539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:00.883154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:00.885401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:00.889970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3a3ad93030477dbf23ea33482be13b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:00.909664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 881/1758] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:00.909664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:00.959527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bd449-42b5-48e2-8808-a566fddddad3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:00.960956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 878/1759] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:00 2022] DELETE /identity/v3/users/0a3a3ad93030477dbf23ea33482be13b => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:00.966067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:01.187076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbe376-7f78-4995-a894-783770f98b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:01.188199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbe376-7f78-4995-a894-783770f98b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.190357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbe376-7f78-4995-a894-783770f98b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.190357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbe376-7f78-4995-a894-783770f98b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.190357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbe376-7f78-4995-a894-783770f98b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:01.192092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbe376-7f78-4995-a894-783770f98b73 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:01.226561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 882/1760] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:01.228230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.228898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.229302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf272b5-9b37-433e-9a64-ab7a43f47e3a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:01.230140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.235037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faa8f5-3c6d-47c4-bb4b-a3cfb80bc9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:01.235775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faa8f5-3c6d-47c4-bb4b-a3cfb80bc9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.250677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faa8f5-3c6d-47c4-bb4b-a3cfb80bc9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.251042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faa8f5-3c6d-47c4-bb4b-a3cfb80bc9dc admin admin] PATH_INFO: `/v3/projects/e3e0de1785444727a55f2b77b4d08d27/users/9996b745cd07419198eac891c110eef3/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.285159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:01.289131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:01.291071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faa8f5-3c6d-47c4-bb4b-a3cfb80bc9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e0de1785444727a55f2b77b4d08d27, user_id=9996b745cd07419198eac891c110eef3, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:01.293841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faa8f5-3c6d-47c4-bb4b-a3cfb80bc9dc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:01.313558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 883/1761] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:01.314587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:01.340139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ecc9a-f052-42c1-9471-fbbc0e8cd7bd None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:01.355298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:01.356942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.357438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.357989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] PATH_INFO: `/v3/projects/e0e51503a67f430ba9c7d27d3e52975d/users/443c4d399e994a4ea5cbb50155bfd535/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.392854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e51503a67f430ba9c7d27d3e52975d, user_id=443c4d399e994a4ea5cbb50155bfd535, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:01.398138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:01.398670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 879/1762] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:00 2022] PUT /identity/v3/projects/e3e0de1785444727a55f2b77b4d08d27/users/9996b745cd07419198eac891c110eef3/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:01.400089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.403879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.403879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.530609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 884/1763] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:01 2022] PUT /identity/v3/projects/e0e51503a67f430ba9c7d27d3e52975d/users/443c4d399e994a4ea5cbb50155bfd535/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:01.542242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:01.546890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:01.631084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:01.639033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:01.695318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7452678-0eff-467b-9861-9ed1835eedda admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:01.710204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d559-3462-405f-9e07-b9314c7c1217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:01.711136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d559-3462-405f-9e07-b9314c7c1217 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.711638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d559-3462-405f-9e07-b9314c7c1217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.712113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d559-3462-405f-9e07-b9314c7c1217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.801307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 880/1764] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:01.805392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:01.872980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:01.909157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d559-3462-405f-9e07-b9314c7c1217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:01.910742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d559-3462-405f-9e07-b9314c7c1217 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:01.946326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 885/1765] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:01.951995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:01.951995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:01.951995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:01.973299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:02.057187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e88a0-bc77-4e5e-944b-a40ca4b31558 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:02.072973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:02.077447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] MFA Rules not processed for user `443c4d399e994a4ea5cbb50155bfd535`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:02.078028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.080122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.081960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:02.084347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:02.086706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:02.093024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:02.140400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 881/1766] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:02.149134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:02.181866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfabb6-5264-486d-85ae-2042325e7173 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:02.195430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e7056-2239-4da7-8f95-9e6735980156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:02.196121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e7056-2239-4da7-8f95-9e6735980156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.196442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e7056-2239-4da7-8f95-9e6735980156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.196708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e7056-2239-4da7-8f95-9e6735980156 admin admin] PATH_INFO: `/v3/projects/e3e0de1785444727a55f2b77b4d08d27/users/9996b745cd07419198eac891c110eef3/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:02.200376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e7056-2239-4da7-8f95-9e6735980156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e0de1785444727a55f2b77b4d08d27, user_id=9996b745cd07419198eac891c110eef3, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:02.203620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e7056-2239-4da7-8f95-9e6735980156 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:02.218108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 882/1767] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:02 2022] PUT /identity/v3/projects/e3e0de1785444727a55f2b77b4d08d27/users/9996b745cd07419198eac891c110eef3/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:02.229690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.230752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.231170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:02.255158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 886/1768] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:02.261154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:02.269696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] MFA Rules not processed for user `9996b745cd07419198eac891c110eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:02.273096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:02.297915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fcf06-4511-4f63-a5c1-2862e2ae7439 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:02.434611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 883/1769] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:02.435833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:02.439217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:02.547551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:02.548378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.548874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.549226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:02.549973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:02.552465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:02.608583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf034d7-3df6-47fa-9664-a1fdfeb726dd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:02.615319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:02.616128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.620186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.620186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:02.729664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 887/1770] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:02 2022] POST /identity/v3/projects => generated 355 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:02.733235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:02.733531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:02.734223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:02.735895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:02.766001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 884/1771] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:46:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:46:02.768820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:02.793127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d9ed-69d4-4a02-984f-4fb2f628d1ed service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:02.935441 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495630-beef-4b43-9791-39f49fdac832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:02.938440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495630-beef-4b43-9791-39f49fdac832 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.938876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495630-beef-4b43-9791-39f49fdac832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.939742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495630-beef-4b43-9791-39f49fdac832 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:02.940629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0495630-beef-4b43-9791-39f49fdac832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:02.948295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0495630-beef-4b43-9791-39f49fdac832 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:02.958091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba81941-d9cb-4618-8c95-f9f55d1c2548 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:02.987030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:02.987676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:02.988050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:02.988252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:03.087330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 885/1772] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:02 2022] POST /identity/v3/users => generated 284 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:03.096070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:03.096703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:03.097971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:03.150103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] MFA Rules not processed for user `b50c8a35e28e44bba3a4b4e56057a3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:03.161952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:03.177401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:03.185691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:03.231698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 888/1773] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:03.234466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:03.284437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f62fc-62a6-43b8-b130-aa91925db24a service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:03.302302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:03.303015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:03.303233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:03.305459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:03.306305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:03.310957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:03.351899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 889/1774] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:03.351899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:03.364752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175900-de10-46c9-b9b9-81847b030a41 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:03.380583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:03.381480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:03.392128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:03.394245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] PATH_INFO: `/v3/projects/bf64ba8b91e94fd09fc6b42026ede7d1/users/15c62c6bf2c24611a8fa3e76777f96b6/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:03.398729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 886/1775] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Sep 7 14:46:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:03.452968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf64ba8b91e94fd09fc6b42026ede7d1, user_id=15c62c6bf2c24611a8fa3e76777f96b6, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:03.456499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:03.603916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 890/1776] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:03 2022] PUT /identity/v3/projects/bf64ba8b91e94fd09fc6b42026ede7d1/users/15c62c6bf2c24611a8fa3e76777f96b6/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:03.610071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:03.642305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:03.645107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:03.672101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac481f-b0f8-42b4-a03b-426d98e4793c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:03.908059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:03.909631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:03.910339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:03.910783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:03.910896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cd0f5-b2b7-4338-b8d9-fc4d80d561f7 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:03.913660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:03.916747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:03.921506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:03.922173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:03.922457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:03.922726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] PATH_INFO: `/v3/projects/ff801b76186c43d1a95720a2eec820f8` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:03.924077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff801b76186c43d1a95720a2eec820f8)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:03.926172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:03.953113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 887/1777] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:03.954060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:03.959307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:04.095266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:04.113717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deacdfe-dc13-4c91-8918-fee8caaf6130 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:04.137872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff801b76186c43d1a95720a2eec820f8', 'request_id': 'req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:04.162214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:04.250285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff801b76186c43d1a95720a2eec820f8', 'request_id': 'req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:04.258059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff801b76186c43d1a95720a2eec820f8', 'request_id': 'req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:04.291835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] The token cache is being invalidate because project ff801b76186c43d1a95720a2eec820f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:04.292936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:04.295967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:04.298458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 891/1778] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:03 2022] DELETE /identity/v3/projects/ff801b76186c43d1a95720a2eec820f8 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:04.299619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:04.301632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:04.340640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:04.341357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:04.341682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:04.342012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:04.355746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:04.439765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:04.446157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:04.514580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 888/1779] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:04.516608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:04.524641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2da9-7a1f-42e3-b2a4-a56ee2b65d87 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:04.527980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:04.546219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3b8f0-9929-413b-91a8-77f88d93f63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:04.547071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3b8f0-9929-413b-91a8-77f88d93f63f admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:04.547453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3b8f0-9929-413b-91a8-77f88d93f63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:04.547876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3b8f0-9929-413b-91a8-77f88d93f63f admin admin] PATH_INFO: `/v3/projects/bf64ba8b91e94fd09fc6b42026ede7d1/users/15c62c6bf2c24611a8fa3e76777f96b6/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:04.555390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3b8f0-9929-413b-91a8-77f88d93f63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf64ba8b91e94fd09fc6b42026ede7d1, user_id=15c62c6bf2c24611a8fa3e76777f96b6, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:04.558866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3b8f0-9929-413b-91a8-77f88d93f63f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:04.582689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 892/1780] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:04 2022] PUT /identity/v3/projects/bf64ba8b91e94fd09fc6b42026ede7d1/users/15c62c6bf2c24611a8fa3e76777f96b6/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:04.597950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:04.598414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:04.598752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:04.608247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875c8ae-2eab-4d77-9885-624ac5852072 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:04.674399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] MFA Rules not processed for user `15c62c6bf2c24611a8fa3e76777f96b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:04.680625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:04.833757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:04.836550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:04.837359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:04.838580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] PATH_INFO: `/v3/users/f19b283699f040a3acbea0bc709eabf5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:04.840956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19b283699f040a3acbea0bc709eabf5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:04.846653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:04.900513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 893/1781] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:04.935803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:04.939496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:05.076625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fb735-ab51-449e-97fd-ab085e71dc10 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:05.094924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:05.095554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:05.095887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:05.096252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:05.137499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:05.139808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:05.191406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19b283699f040a3acbea0bc709eabf5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:05.195618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19b283699f040a3acbea0bc709eabf5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:05.199373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19b283699f040a3acbea0bc709eabf5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:05.229221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 894/1782] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:05.346099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19b283699f040a3acbea0bc709eabf5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:05.362468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19b283699f040a3acbea0bc709eabf5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:05.442031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 889/1783] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:04 2022] DELETE /identity/v3/users/f19b283699f040a3acbea0bc709eabf5 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:05.571615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:05.581870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fc569-32fc-4e9a-ad1d-70f3d188ebce service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:05.782816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:05.785546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:05.786837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:05.787886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:05.799327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:05.859252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:05.942329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:05.943853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5af526-ea0e-4af7-b7e6-b10516b4d687 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:05.971979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 895/1784] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:06.080457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:07.175391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:07.803235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:07.917184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:07.919528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:08.095449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab3617-d778-4c59-8d71-c59433a89f03 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:08.112565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:08.116048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.117378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.119403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] PATH_INFO: `/v3/projects/fc824547a9684d11924f2f30233caa10` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.128075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc824547a9684d11924f2f30233caa10)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:08.139229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:08.353876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc824547a9684d11924f2f30233caa10', 'request_id': 'req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:08.454937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc824547a9684d11924f2f30233caa10', 'request_id': 'req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:08.466478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc824547a9684d11924f2f30233caa10', 'request_id': 'req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:08.506212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] The token cache is being invalidate because project fc824547a9684d11924f2f30233caa10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:08.506212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:08.506212 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-43ab976b-0d4a-4d3b-921b-4c31be3dbc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:08.510529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 890/1785] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:07 2022] DELETE /identity/v3/projects/fc824547a9684d11924f2f30233caa10 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:08.515378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.516091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.516790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.557464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:08.560936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:08.697930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 896/1786] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:08.706597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06ce1-cc3f-41e1-ae12-23f0650aaaad None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.706880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06ce1-cc3f-41e1-ae12-23f0650aaaad None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.707158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06ce1-cc3f-41e1-ae12-23f0650aaaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.743424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-65f06ce1-cc3f-41e1-ae12-23f0650aaaad None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:08.744686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-65f06ce1-cc3f-41e1-ae12-23f0650aaaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:08.787799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 891/1787] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:08.788045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:08.801663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e64c-13a7-40ba-a950-cf94c615ade6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:08.812315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:08.844705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d3381-b454-4dce-b784-df96bb15439d None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.845005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d3381-b454-4dce-b784-df96bb15439d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.845277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d3381-b454-4dce-b784-df96bb15439d None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.847444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 892/1788] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:46:08.858774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.858774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.858774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.865460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:08.865857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:08.867554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:08.894805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 897/1789] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:08.902162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.902442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.902738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.903867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:08.917169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:08.920675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c005e87-f599-41ed-bfb1-09c10ee5cfd2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:08.927683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:08.929091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:08.929615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:08.930141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:08.930993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:08.933541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:09.000707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:09.006159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:09.056487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:09.095555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 898/1790] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:08 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:09.101912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:09.213772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 893/1791] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:09.216174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:09.218216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:09.239938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1025120-ac4b-4481-a823-912afca5f41d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:09.247232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:09.248430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.249018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.249616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.259966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6abf9f-0339-40f6-bab8-707892edffc5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:09.261298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:09.263725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:09.282570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:09.283676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.283972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.284339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] PATH_INFO: `/v3/users/3036bb7a137741d7ac67821d0e7919ca` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.288450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3036bb7a137741d7ac67821d0e7919ca)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:09.295795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:09.353809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 899/1792] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:09 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:09.355929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:09.385768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e1371-9441-4e21-9f47-9cd78332ec2c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:09.385768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:09.385768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.385768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.386054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.386054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:09.386054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:09.398986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 900/1793] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:09.403374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:09.440337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341db68-cd91-4bf9-8056-0f2d814f02bb admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:09.454964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:09.455784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.459382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.461064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.462123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:09.467239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:09.506959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 901/1794] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:09.507940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:09.530139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:09.561861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3036bb7a137741d7ac67821d0e7919ca'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:09.563375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3036bb7a137741d7ac67821d0e7919ca'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:09.564667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3036bb7a137741d7ac67821d0e7919ca'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:09.693784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3036bb7a137741d7ac67821d0e7919ca'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:09.705459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3036bb7a137741d7ac67821d0e7919ca'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:09.732389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9b943-5151-40d1-b02b-0f433a68afe3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:09.750358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 894/1795] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:09 2022] DELETE /identity/v3/users/3036bb7a137741d7ac67821d0e7919ca => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:09.750358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:09.750358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:09.769124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:09.772702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.772702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.772702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.785273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:09.798220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:09.822403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:09.828506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d03c4a-763a-4cd3-8130-2a6d7854caf0 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:09.872475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 902/1796] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:09.874258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522ee2f-97f6-41f6-bfce-56c1ae308883 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.874767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522ee2f-97f6-41f6-bfce-56c1ae308883 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.876232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522ee2f-97f6-41f6-bfce-56c1ae308883 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.880968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 903/1797] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:46:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:46:09.885973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c230f-ddd0-4b5e-8046-969e340b817d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.885973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c230f-ddd0-4b5e-8046-969e340b817d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.885973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c230f-ddd0-4b5e-8046-969e340b817d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.920642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf715f21-070f-4813-9612-edd26f6186c4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:09.937478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7c5c230f-ddd0-4b5e-8046-969e340b817d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:09.941507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7c5c230f-ddd0-4b5e-8046-969e340b817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:09.946746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8d742-2153-496f-b72a-b7e5ee9b33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:09.949128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8d742-2153-496f-b72a-b7e5ee9b33db admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.949240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8d742-2153-496f-b72a-b7e5ee9b33db admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.949463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8d742-2153-496f-b72a-b7e5ee9b33db admin admin] PATH_INFO: `/v3/projects/d0c97fb5c44e4b2b93af30e753ceea0f/users/de3e20efa946424ab873a94ff6de37ce/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.968091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 904/1798] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:09.972400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23225c06-e000-4715-8def-84decd0ac75d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:09.972806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23225c06-e000-4715-8def-84decd0ac75d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:09.973015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23225c06-e000-4715-8def-84decd0ac75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:09.993084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8d742-2153-496f-b72a-b7e5ee9b33db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c97fb5c44e4b2b93af30e753ceea0f, user_id=de3e20efa946424ab873a94ff6de37ce, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:09.995793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8d742-2153-496f-b72a-b7e5ee9b33db admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:10.024652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-23225c06-e000-4715-8def-84decd0ac75d None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:10.026011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-23225c06-e000-4715-8def-84decd0ac75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:10.148597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 905/1799] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:10.149634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23225c06-e000-4715-8def-84decd0ac75d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:10.156904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-23225c06-e000-4715-8def-84decd0ac75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:10.161236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 895/1800] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:09 2022] PUT /identity/v3/projects/d0c97fb5c44e4b2b93af30e753ceea0f/users/de3e20efa946424ab873a94ff6de37ce/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:10.172575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.180587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.180884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.237621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] MFA Rules not processed for user `de3e20efa946424ab873a94ff6de37ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:10.239269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:10.284563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23225c06-e000-4715-8def-84decd0ac75d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:10.350159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:10.351326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.351611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.352026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.357274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 896/1801] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:10.358908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:10.361676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:10.374351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:10.426069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:10.427597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:10.438709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 906/1802] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:10.453186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:10.460358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67c42-d81b-4dad-a0fc-0899f57be4c8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:10.472488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:10.473293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.473949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.474411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.479099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a21f81-20fe-4d31-b46c-dbf9ba6275a0 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:10.490144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:10.491451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:10.492010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:10.492122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.492342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.492617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.494265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:10.498612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:10.511934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 897/1803] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:10.633407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 907/1804] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:10 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:10.640467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:10.655348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599cbd3-62f9-41d1-b698-c605f03be749 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:10.745231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:10.745858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.746142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.746410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.747044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:10.757686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:10.878570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 898/1805] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:10 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:10.885076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:10.896527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0d7a-a2f1-44c7-883a-f3addb6c73f4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:10.905070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:10.905941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.906247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.906615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.907552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:10.909371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:10.921808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 908/1806] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:10.926176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:10.936941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29d6ff-5c8f-4de0-82f8-7c5fc18953fb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:10.941730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:10.942390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:10.942666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:10.942939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] PATH_INFO: `/v3/projects/dbdc0bcd75304cc5a5501f7970bffe91/users/4c9342488fd54923b0df1bdb818c2c05/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:10.958550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdc0bcd75304cc5a5501f7970bffe91, user_id=4c9342488fd54923b0df1bdb818c2c05, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:10.961482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:11.072494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:11.074274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:11.076324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 899/1807] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:10 2022] PUT /identity/v3/projects/dbdc0bcd75304cc5a5501f7970bffe91/users/4c9342488fd54923b0df1bdb818c2c05/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:11.088385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:11.094663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ffdc-d85d-4d45-8845-9cd619fcd644 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:11.198356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:11.199081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:11.200198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:11.200198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] PATH_INFO: `/v3/projects/2d3e14e5745e40b4ab848e0a2bb241bc` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:11.200961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d3e14e5745e40b4ab848e0a2bb241bc)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:11.207973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:11.252078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9a20d-4df9-4c75-98c2-6b42213b1d75 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:11.257817 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:11.258691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:11.259189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:11.259657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:11.260539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:11.262672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:11.277084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 900/1808] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:11.280911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:11.299686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03cf7-8e5f-477a-9ce0-c40e476bceeb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:11.305451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec89ad3-25d6-4620-b075-1f0d5dd6c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:11.306263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec89ad3-25d6-4620-b075-1f0d5dd6c5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:11.306538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec89ad3-25d6-4620-b075-1f0d5dd6c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:11.306806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec89ad3-25d6-4620-b075-1f0d5dd6c5ea admin admin] PATH_INFO: `/v3/projects/dbdc0bcd75304cc5a5501f7970bffe91/users/4c9342488fd54923b0df1bdb818c2c05/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:11.309070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec89ad3-25d6-4620-b075-1f0d5dd6c5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdc0bcd75304cc5a5501f7970bffe91, user_id=4c9342488fd54923b0df1bdb818c2c05, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:11.311897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec89ad3-25d6-4620-b075-1f0d5dd6c5ea admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:11.324100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 901/1809] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:11 2022] PUT /identity/v3/projects/dbdc0bcd75304cc5a5501f7970bffe91/users/4c9342488fd54923b0df1bdb818c2c05/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:11.336137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:11.336605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:11.336879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:11.341747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d3e14e5745e40b4ab848e0a2bb241bc', 'request_id': 'req-e0a5eac7-770c-47f9-b033-cf83d8477ea4'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.388049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] MFA Rules not processed for user `4c9342488fd54923b0df1bdb818c2c05`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:11.388049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:11.419588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d3e14e5745e40b4ab848e0a2bb241bc', 'request_id': 'req-e0a5eac7-770c-47f9-b033-cf83d8477ea4'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.426211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d3e14e5745e40b4ab848e0a2bb241bc', 'request_id': 'req-e0a5eac7-770c-47f9-b033-cf83d8477ea4'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.452049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] The token cache is being invalidate because project 2d3e14e5745e40b4ab848e0a2bb241bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:11.452049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.452991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.457740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 909/1810] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:11 2022] DELETE /identity/v3/projects/2d3e14e5745e40b4ab848e0a2bb241bc => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:11.458818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:11.461111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:11.499637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 902/1811] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:11.521459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:11.526072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:11.565801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5eac7-770c-47f9-b033-cf83d8477ea4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:11.573202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:11.573844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:11.574119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:11.574401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] PATH_INFO: `/v3/users/9996b745cd07419198eac891c110eef3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:11.575677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9996b745cd07419198eac891c110eef3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:11.578641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:11.603752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:11.623137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa256cb3-222d-4ac5-afd3-9ca1ce3088ab None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:11.627978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:11.628634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:11.629171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:11.629431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:11.639832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:11.641315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:11.653085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 903/1812] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:11.825545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9996b745cd07419198eac891c110eef3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.911132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9996b745cd07419198eac891c110eef3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.923092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9996b745cd07419198eac891c110eef3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.925015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9996b745cd07419198eac891c110eef3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.925739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282ee328-2c5c-4489-9187-28c4626ac8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9996b745cd07419198eac891c110eef3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:11.940086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:11.956701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 910/1813] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:11 2022] DELETE /identity/v3/users/9996b745cd07419198eac891c110eef3 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:12.458260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:12.474435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb781ac-46ca-421e-b5df-e535b00e76ef service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:12.572785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:12.572785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:12.572785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:12.572785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] PATH_INFO: `/v3/projects/e3e0de1785444727a55f2b77b4d08d27` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:12.573324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:12.574034 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e0de1785444727a55f2b77b4d08d27)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:12.576000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:12.696995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e0de1785444727a55f2b77b4d08d27', 'request_id': 'req-f893d9de-646b-4d1e-bdf3-58aa7766733f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:12.798467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e0de1785444727a55f2b77b4d08d27', 'request_id': 'req-f893d9de-646b-4d1e-bdf3-58aa7766733f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:12.809555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e0de1785444727a55f2b77b4d08d27', 'request_id': 'req-f893d9de-646b-4d1e-bdf3-58aa7766733f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:12.841780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] The token cache is being invalidate because project e3e0de1785444727a55f2b77b4d08d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:12.842483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:12.846873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:12.856806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 904/1814] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:12 2022] DELETE /identity/v3/projects/e3e0de1785444727a55f2b77b4d08d27 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:12.866430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1549-f8ae-4a20-9352-fda0ba38f14b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:12.866909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1549-f8ae-4a20-9352-fda0ba38f14b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:12.867378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1549-f8ae-4a20-9352-fda0ba38f14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:12.891652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:12.897349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:12.903879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b48a1549-f8ae-4a20-9352-fda0ba38f14b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:12.905556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b48a1549-f8ae-4a20-9352-fda0ba38f14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:12.999354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 911/1815] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:13.002242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.002481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.003094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.030907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893d9de-646b-4d1e-bdf3-58aa7766733f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.038070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c9342488fd54923b0df1bdb818c2c05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbdc0bcd75304cc5a5501f7970bffe91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.039602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.040095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.040429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:13.040860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.041705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:13.042128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:13.042627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.044505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.050682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 912/1816] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:13.055818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.067258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 905/1817] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:46:13.071026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cc99a-66a8-4354-98e7-44f01b15c251 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.073500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.077728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.078788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.079546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.080436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.081183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:13.082453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.083570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30917f-29fe-43b7-96bc-014c06f07991 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.085778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.095157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c9342488fd54923b0df1bdb818c2c05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbdc0bcd75304cc5a5501f7970bffe91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.097546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.097828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.098097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.098754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.100404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 913/1818] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:13.101157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.112680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.136808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99481d-c12e-4c5e-a9d1-21d8499af03e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.146514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.147787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.148075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.148512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.149453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.155366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.204504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 906/1819] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:13 2022] POST /identity/v3/projects => generated 354 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:13.207206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.276058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 914/1820] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:13 2022] POST /identity/v3/projects => generated 393 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:13.279928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.427447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74aa4c-8d87-4b91-bd63-ae577e847cb4 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.479954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134f432-f7a7-41c1-ba69-af1b862e6033 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.501323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.502628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.503541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.504262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.505618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.508928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.572847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c9342488fd54923b0df1bdb818c2c05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbdc0bcd75304cc5a5501f7970bffe91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.573897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.574398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.574934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.575938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:13.576516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.578646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.592405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 907/1821] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:46:13.612825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 915/1822] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:13 2022] POST /identity/v3/users => generated 297 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:13.618846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.644256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7126-dac9-43b1-bbc5-07c2a98bf814 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.650785 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.655763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.656851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.657490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.657961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.659487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.661864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.674639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6e0fb-0b6f-4532-badf-36bf5b903615 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.681670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c9342488fd54923b0df1bdb818c2c05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbdc0bcd75304cc5a5501f7970bffe91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.683535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.684041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.685013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] PATH_INFO: `/v3/projects/74e4c9832ee84264b2c761864409eb2d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.686715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorizing `identity:delete_project(project_id=74e4c9832ee84264b2c761864409eb2d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.688816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 908/1823] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:13.699526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.701808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.717123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693a1c-0597-4304-a2c6-6a70b91167ae admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.724490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.724490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.724490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.724490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.730583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.730583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.748780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 909/1824] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:13.750344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:13.776099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe07089-f5bf-4985-aa35-70d17d2f11de admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:13.776099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363d96e-9055-4de0-bc72-7274c979ec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:13.776099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363d96e-9055-4de0-bc72-7274c979ec2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.776099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363d96e-9055-4de0-bc72-7274c979ec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.778266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363d96e-9055-4de0-bc72-7274c979ec2d admin admin] PATH_INFO: `/v3/projects/470c523ea85b41e4b62142b392d34131/users/e940cf833a244f40ab739328f88a7afb/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.800844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363d96e-9055-4de0-bc72-7274c979ec2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=470c523ea85b41e4b62142b392d34131, user_id=e940cf833a244f40ab739328f88a7afb, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:13.803673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363d96e-9055-4de0-bc72-7274c979ec2d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:13.906851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 910/1825] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:13 2022] PUT /identity/v3/projects/470c523ea85b41e4b62142b392d34131/users/e940cf833a244f40ab739328f88a7afb/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:13.926116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e4c9832ee84264b2c761864409eb2d', 'request_id': 'req-5f68323f-941e-4a1e-99a3-4854ccad9cc1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:13.927543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:13.927543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:13.927543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:13.958327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] MFA Rules not processed for user `e940cf833a244f40ab739328f88a7afb`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:13.960266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:14.001557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e4c9832ee84264b2c761864409eb2d', 'request_id': 'req-5f68323f-941e-4a1e-99a3-4854ccad9cc1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.009212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e4c9832ee84264b2c761864409eb2d', 'request_id': 'req-5f68323f-941e-4a1e-99a3-4854ccad9cc1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.022472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] The token cache is being invalidate because project 74e4c9832ee84264b2c761864409eb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:14.022682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.023352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.029852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 916/1826] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:13 2022] DELETE /identity/v3/projects/74e4c9832ee84264b2c761864409eb2d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:14.059472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 911/1827] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:14.059679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.059679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:14.094282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:14.117632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68323f-941e-4a1e-99a3-4854ccad9cc1 tempest-VolumeQuotaClassesTest-1245757162 tempest-VolumeQuotaClassesTest-1245757162-project-admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.122329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.122930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.123206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.123473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.124098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.126044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.208705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 917/1828] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:14 2022] POST /identity/v3/projects => generated 391 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:14.213083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.223957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bb7ef-fc1f-43df-a38c-59b47c0d895a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.280013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.280600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.280881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.281149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.281785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.283677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.397370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 912/1829] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:14 2022] POST /identity/v3/users => generated 302 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:14.401461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.411822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad66ee-8b8c-447c-8713-abdfdf2bbd2d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.416993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.417603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.417884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.418291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.419409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.421096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.434140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 918/1830] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:14.438509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.448503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc1168-5b14-4f20-b59f-90e164a52693 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.453150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.453806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.454086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.454359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] PATH_INFO: `/v3/projects/12c371c8075e420fa3abd4a99e093b66/users/f0151ebe40d14542aa6d580cdc6192d2/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.469334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c371c8075e420fa3abd4a99e093b66, user_id=f0151ebe40d14542aa6d580cdc6192d2, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.472069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.603638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 913/1831] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:14 2022] PUT /identity/v3/projects/12c371c8075e420fa3abd4a99e093b66/users/f0151ebe40d14542aa6d580cdc6192d2/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:14.608222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.618846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdeb92-574b-4522-9838-bafb41d5d060 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.677685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.678855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.679137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.679411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.680057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.681704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.697258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 919/1832] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:14.699867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.700473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:14.704284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:14.707245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.716627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea06401-1d46-42b5-bcc9-0bd7a6ce88a4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.717972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58592f8-871c-48db-bca5-4785b2c9fe5a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.721861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.722573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.723453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.723558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b20bc-1aad-4e89-9826-ac2b3da1729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.723660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] PATH_INFO: `/v3/users/de3e20efa946424ab873a94ff6de37ce` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.723865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b20bc-1aad-4e89-9826-ac2b3da1729e admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.724129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b20bc-1aad-4e89-9826-ac2b3da1729e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.725474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b20bc-1aad-4e89-9826-ac2b3da1729e admin admin] PATH_INFO: `/v3/projects/12c371c8075e420fa3abd4a99e093b66/users/f0151ebe40d14542aa6d580cdc6192d2/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.725590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3e20efa946424ab873a94ff6de37ce)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.727293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b20bc-1aad-4e89-9826-ac2b3da1729e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c371c8075e420fa3abd4a99e093b66, user_id=f0151ebe40d14542aa6d580cdc6192d2, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.727978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.730279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b20bc-1aad-4e89-9826-ac2b3da1729e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.743206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 920/1833] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:14 2022] PUT /identity/v3/projects/12c371c8075e420fa3abd4a99e093b66/users/f0151ebe40d14542aa6d580cdc6192d2/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:14.754942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.756329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.756329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.793484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] MFA Rules not processed for user `f0151ebe40d14542aa6d580cdc6192d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:14.794886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:14.873495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 921/1834] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:14.884488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:14.885888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:14.931132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3e20efa946424ab873a94ff6de37ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.932040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3e20efa946424ab873a94ff6de37ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.932760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3e20efa946424ab873a94ff6de37ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:14.945860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21cba5-d74d-4343-b8e2-63e26bc6e04d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:14.949627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:14.950264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:14.950542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:14.950811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:14.952648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:14.969262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:14.969262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:14.978870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 922/1835] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:15.002322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3e20efa946424ab873a94ff6de37ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:15.011296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d239f131-92db-423e-b731-06eb2f97d057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3e20efa946424ab873a94ff6de37ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:15.035574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:15.038441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:15.039266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 914/1836] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:14 2022] DELETE /identity/v3/users/de3e20efa946424ab873a94ff6de37ce => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:15.043425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:15.043712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:15.043987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:15.050549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9407c-9c95-48e5-b7dc-70b263d2b274 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:15.066952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:15.068248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:15.105886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:15.106478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:15.106761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:15.107035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:15.109138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:15.152710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 915/1837] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:15.187199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:15.187199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:15.200223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:15.201792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:15.237277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 923/1838] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:15.318070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731c326-07e5-48d0-8228-d62c5ace043d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:15.322442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:15.323164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:15.323542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:15.323909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:15.349514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:15.351618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:15.363901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 916/1839] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:15.493534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:15.852649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:15.854246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:15.879372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3bbe-b381-4c01-ae5e-e23157b3120c service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:15.902984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:15.903903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:15.904356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:15.904808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] PATH_INFO: `/v3/projects/d0c97fb5c44e4b2b93af30e753ceea0f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:15.906464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c97fb5c44e4b2b93af30e753ceea0f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:15.912185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:16.112910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c97fb5c44e4b2b93af30e753ceea0f', 'request_id': 'req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.236569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c97fb5c44e4b2b93af30e753ceea0f', 'request_id': 'req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.247396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c97fb5c44e4b2b93af30e753ceea0f', 'request_id': 'req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.268314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] The token cache is being invalidate because project d0c97fb5c44e4b2b93af30e753ceea0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:16.268667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.269460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.270952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 924/1840] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:15 2022] DELETE /identity/v3/projects/d0c97fb5c44e4b2b93af30e753ceea0f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:16.276677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:16.297198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22f3a-33a8-4fc9-9a7a-eea0ad2f3b81 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:16.375165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:16.436198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:16.437136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:16.437712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:16.438185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] PATH_INFO: `/v3/users/4c9342488fd54923b0df1bdb818c2c05` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:16.439563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9342488fd54923b0df1bdb818c2c05)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:16.442273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:16.637625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9342488fd54923b0df1bdb818c2c05'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.641299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9342488fd54923b0df1bdb818c2c05'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.644236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9342488fd54923b0df1bdb818c2c05'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.734182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9342488fd54923b0df1bdb818c2c05'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.744982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c22e0c49-a016-4a59-bc80-ffefae9f0fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9342488fd54923b0df1bdb818c2c05'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:16.769528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 917/1841] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:16 2022] DELETE /identity/v3/users/4c9342488fd54923b0df1bdb818c2c05 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:17.484980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:18.084439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:18.084669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:18.109987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b09a4f-61b1-43db-bec9-c2b647cbccdc admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:18.151137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:18.183373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:18.183989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:18.184321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:18.184595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] PATH_INFO: `/v3/projects/dbdc0bcd75304cc5a5501f7970bffe91` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:18.186473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdc0bcd75304cc5a5501f7970bffe91)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:18.187985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:18.335090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdc0bcd75304cc5a5501f7970bffe91', 'request_id': 'req-33e1fdcf-3b82-4044-99fe-6b6027d84fec'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:18.408021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdc0bcd75304cc5a5501f7970bffe91', 'request_id': 'req-33e1fdcf-3b82-4044-99fe-6b6027d84fec'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:18.415731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdc0bcd75304cc5a5501f7970bffe91', 'request_id': 'req-33e1fdcf-3b82-4044-99fe-6b6027d84fec'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:18.430415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] The token cache is being invalidate because project dbdc0bcd75304cc5a5501f7970bffe91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:18.431341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:18.431485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-33e1fdcf-3b82-4044-99fe-6b6027d84fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:18.436472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 925/1842] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:18 2022] DELETE /identity/v3/projects/dbdc0bcd75304cc5a5501f7970bffe91 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:18.438492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:18.439413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:18.439994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:18.464017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:18.465493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:18.532514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 918/1843] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:18.538434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:18.538917 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:18.539051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:18.559087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:18.561000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:18.568412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 926/1844] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:18.572816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:18.582749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5f95-8d78-45ab-b570-4bc06d8a87d5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:18.588257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:18.588829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:18.589109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:18.589385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:18.590040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:18.590399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:18.592091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:18.608333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 919/1845] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:18.610300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:18.644430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b74c90-04ad-41eb-87f7-2f569834f0e2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:18.666277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:18.667100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:18.667542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:18.668124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:18.669186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:18.671473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:18.734050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:18.844758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 927/1846] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:18 2022] POST /identity/v3/projects => generated 401 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:18.848388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:18.865690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d2550-2848-4bed-bc77-4e6c4a220f0c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:18.962589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:18.963424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:18.963834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:18.966217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:18.967124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:18.969470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.066304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 920/1847] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:18 2022] POST /identity/v3/users => generated 301 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:19.073464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.085328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec6a8d-d0c6-4f21-8797-10aa662cc20a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.093215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.094234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.094727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.095205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.096099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.097993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.115019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.120382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 928/1848] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:19.126485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348171-a358-4ae3-a700-33e367f3b10b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.132878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe4e62-f205-49de-a84e-33ff685ff22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.133734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe4e62-f205-49de-a84e-33ff685ff22e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.134194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe4e62-f205-49de-a84e-33ff685ff22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.134699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe4e62-f205-49de-a84e-33ff685ff22e admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.135497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe4e62-f205-49de-a84e-33ff685ff22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.137301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe4e62-f205-49de-a84e-33ff685ff22e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.150338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 921/1849] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:19.154339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.164686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0bd37-c36e-4390-94a2-ddfb11e06765 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.177837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.177837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.177837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.177837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] PATH_INFO: `/v3/projects/9ec611a4c22a48ce80e77b85cb0bea84/users/87f912b9a1dd4aeb8f6b3b0ce398483c/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.202240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec611a4c22a48ce80e77b85cb0bea84, user_id=87f912b9a1dd4aeb8f6b3b0ce398483c, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.202240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.317312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 929/1850] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:19 2022] PUT /identity/v3/projects/9ec611a4c22a48ce80e77b85cb0bea84/users/87f912b9a1dd4aeb8f6b3b0ce398483c/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:19.327789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.328075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.328388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.366280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] MFA Rules not processed for user `87f912b9a1dd4aeb8f6b3b0ce398483c`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:19.367530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:19.420357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:19.444388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 922/1851] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:19.449043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.531499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c07957-4faf-41f3-a943-1bb5af8f4f08 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.537368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.539014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.540728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.541254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.542826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.546198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.670444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 930/1852] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:19 2022] POST /identity/v3/projects => generated 399 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:19.674873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.686370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153454-2f69-4c0e-b007-dda8da279baf None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.787902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.788549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.788827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.789100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.789792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.791720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.888582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 923/1853] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:19 2022] POST /identity/v3/users => generated 306 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:19.893521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.903863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a89c-b8c5-4ade-a192-3623e7dde583 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.915414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.915414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.915414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.915414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.915821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.915821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:19.923598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 931/1854] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:19.927718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:19.938821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd98795-9ec0-4566-8723-3991b810b755 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:19.943618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:19.944373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:19.944650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:19.944915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] PATH_INFO: `/v3/projects/9f05c0e1acf7453ab4224960e2392391/users/84555ebf495149e198bf8c8abd687305/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:19.959314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f05c0e1acf7453ab4224960e2392391, user_id=84555ebf495149e198bf8c8abd687305, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:19.962145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:20.031603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 924/1855] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:19 2022] PUT /identity/v3/projects/9f05c0e1acf7453ab4224960e2392391/users/84555ebf495149e198bf8c8abd687305/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:20.035742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:20.048795 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d261b3-60ef-4179-bb85-a507331ec04c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:20.109148 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c559048-60a2-4015-9061-c495d3c7ccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:20.109770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559048-60a2-4015-9061-c495d3c7ccc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:20.110066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559048-60a2-4015-9061-c495d3c7ccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:20.110326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559048-60a2-4015-9061-c495d3c7ccc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:20.110979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c559048-60a2-4015-9061-c495d3c7ccc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:20.112685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c559048-60a2-4015-9061-c495d3c7ccc3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:20.123503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 932/1856] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:20.127784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:20.139796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49edeb-21cc-494f-91f5-a8c9388d6a3a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:20.145256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:20.146013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:20.146296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:20.146574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] PATH_INFO: `/v3/projects/9f05c0e1acf7453ab4224960e2392391/users/84555ebf495149e198bf8c8abd687305/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:20.149493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f05c0e1acf7453ab4224960e2392391, user_id=84555ebf495149e198bf8c8abd687305, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:20.152203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:20.163730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 925/1857] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:20 2022] PUT /identity/v3/projects/9f05c0e1acf7453ab4224960e2392391/users/84555ebf495149e198bf8c8abd687305/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:20.174741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:20.175027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:20.175336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:20.204518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] MFA Rules not processed for user `84555ebf495149e198bf8c8abd687305`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:20.205744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:20.291267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 933/1858] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:20.302024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:20.304741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:20.401263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381f304-f127-47a9-9321-084d0d99d40f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:20.407357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:20.408370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:20.412941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:20.412941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:20.423224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:20.427384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:20.443011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 926/1859] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:20.739628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:20.753557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92610c-3876-4f3b-ac90-90a31492a7da None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:20.760077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392b3d7-1a3f-441e-9da9-47e6f30966fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:20.761122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b3d7-1a3f-441e-9da9-47e6f30966fe service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:20.762206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b3d7-1a3f-441e-9da9-47e6f30966fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:20.762646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b3d7-1a3f-441e-9da9-47e6f30966fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:20.805282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:20.839895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392b3d7-1a3f-441e-9da9-47e6f30966fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:20.841742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392b3d7-1a3f-441e-9da9-47e6f30966fe service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:20.854055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 934/1860] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:20 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:20.905325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:21.195512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:21.197185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:21.316355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28f24f-9a31-4af5-9ac0-b2090594093d service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:21.323922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:21.325914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:21.325914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:21.325914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] PATH_INFO: `/v3/users/e940cf833a244f40ab739328f88a7afb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:21.327482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e940cf833a244f40ab739328f88a7afb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:21.330245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:21.570423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e940cf833a244f40ab739328f88a7afb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:21.571177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e940cf833a244f40ab739328f88a7afb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:21.571792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e940cf833a244f40ab739328f88a7afb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:21.636406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e940cf833a244f40ab739328f88a7afb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:21.644488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e940cf833a244f40ab739328f88a7afb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:21.662634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 927/1861] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:21 2022] DELETE /identity/v3/users/e940cf833a244f40ab739328f88a7afb => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:21.667239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:21.667632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:21.667997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:21.689081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:21.692968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:21.756633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 935/1862] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:21.763487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:21.765002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:21.826023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d719-4821-4c81-be51-0cec6bb51a0c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:21.829646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:21.830256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:21.830535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:21.830805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:21.839756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:21.841301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:21.850636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 928/1863] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:23.256249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:23.259554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:23.274910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618fe6a-e578-4842-9103-d976bcdb11b8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:23.280252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:23.281069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:23.281550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:23.281979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] PATH_INFO: `/v3/projects/470c523ea85b41e4b62142b392d34131` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:23.283145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=470c523ea85b41e4b62142b392d34131)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:23.285153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:23.419172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470c523ea85b41e4b62142b392d34131', 'request_id': 'req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.477723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470c523ea85b41e4b62142b392d34131', 'request_id': 'req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.486983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470c523ea85b41e4b62142b392d34131', 'request_id': 'req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.504642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] The token cache is being invalidate because project 470c523ea85b41e4b62142b392d34131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:23.505079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.506472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-178366ff-47a3-4b79-bf39-d9b5c2ad7d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.511380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 936/1864] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:23 2022] DELETE /identity/v3/projects/470c523ea85b41e4b62142b392d34131 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:23.512633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:23.518290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b8b35-8510-409e-a0f7-2f423bdba498 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:23.518798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b8b35-8510-409e-a0f7-2f423bdba498 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:23.519174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b8b35-8510-409e-a0f7-2f423bdba498 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:23.521657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 937/1865] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:46:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:46:23.526822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e221a01-995e-42af-81be-e0cbf51e9b96 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:23.541501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:23.554578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:23.555381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:23.557084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:23.588672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:23.590608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:23.630645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:23.635973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:23.635973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:23.635973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] PATH_INFO: `/v3/users/f0151ebe40d14542aa6d580cdc6192d2` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:23.635973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0151ebe40d14542aa6d580cdc6192d2)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:23.637325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:23.685002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 938/1866] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:46:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:23.733158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:23.735506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:23.751098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cedb8d-fdb6-4be1-be61-96d19e5c2c73 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:23.757310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:23.758145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:23.758436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:23.758710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:23.769540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:23.771277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:23.783002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 939/1867] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:23.847477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0151ebe40d14542aa6d580cdc6192d2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.849717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0151ebe40d14542aa6d580cdc6192d2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.850385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0151ebe40d14542aa6d580cdc6192d2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.926336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0151ebe40d14542aa6d580cdc6192d2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.934686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-054b2e6f-001a-404a-98a9-ac99dd823312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0151ebe40d14542aa6d580cdc6192d2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:23.958123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 929/1868] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:23 2022] DELETE /identity/v3/users/f0151ebe40d14542aa6d580cdc6192d2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:24.266237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:24.413407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:24.413407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:24.439943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9ccf-4eca-4ab8-bdfb-0b8683316d9a service nova] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:24.518924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:24.519623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:24.519904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:24.520219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] PATH_INFO: `/v3/projects/12c371c8075e420fa3abd4a99e093b66` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:24.522149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c371c8075e420fa3abd4a99e093b66)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:24.524866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:24.692183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c371c8075e420fa3abd4a99e093b66', 'request_id': 'req-c505ed78-63d8-491c-af6b-1a9cc233f34b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:24.826062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c371c8075e420fa3abd4a99e093b66', 'request_id': 'req-c505ed78-63d8-491c-af6b-1a9cc233f34b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:24.835913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c371c8075e420fa3abd4a99e093b66', 'request_id': 'req-c505ed78-63d8-491c-af6b-1a9cc233f34b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:24.852728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] The token cache is being invalidate because project 12c371c8075e420fa3abd4a99e093b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:24.853020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:24.854067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c505ed78-63d8-491c-af6b-1a9cc233f34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:24.857882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 940/1869] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:24 2022] DELETE /identity/v3/projects/12c371c8075e420fa3abd4a99e093b66 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:24.867590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:24.868498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:24.869105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:24.895208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:24.897638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:24.992679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 930/1870] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:24.994875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:24.995234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:24.995588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.033134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:25.034519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:25.052269 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 941/1871] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:25.057290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.067736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44184b0a-9643-405d-97f3-87c895c001d9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.073347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.075688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.075688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.075688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.075688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:25.075688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.077475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.102663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 931/1872] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:25.107308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.118442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23c745-803f-42bd-8b21-84c0f6980a15 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.123573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.124287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.124717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.124998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.125645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.127578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.228237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 942/1873] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:25 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:25.232237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.257456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338ea4b-21bf-4c23-a90e-aee0d8194a29 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.319828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.320477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.320760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.321029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.321657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.323562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.420070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 932/1874] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:25 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:25.424562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.434864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbf625-2e4c-4327-90ba-c45f382cb91f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.439419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.440011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.440341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.440641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.441283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.442894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.453086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 943/1875] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:25.456972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.467472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1235c8-5dfb-412f-a670-98bdbf75cfb2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.475369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1d084-e878-4c40-a1af-69d486ea883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.475369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1d084-e878-4c40-a1af-69d486ea883c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.475369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1d084-e878-4c40-a1af-69d486ea883c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.475369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1d084-e878-4c40-a1af-69d486ea883c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.475721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1d084-e878-4c40-a1af-69d486ea883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.476590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1d084-e878-4c40-a1af-69d486ea883c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.488789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 933/1876] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:25.492943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.502918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b41a9-c377-470f-935c-010987dc6032 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.507902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.508707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.508993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.509538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] PATH_INFO: `/v3/projects/eccd05d89a0949b48f3ab118b66e86ab/users/270b151f93aa49a09a8c9a5fb6ddebb6/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.523236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:25.527685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccd05d89a0949b48f3ab118b66e86ab, user_id=270b151f93aa49a09a8c9a5fb6ddebb6, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.530417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.627151 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 944/1877] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:25 2022] PUT /identity/v3/projects/eccd05d89a0949b48f3ab118b66e86ab/users/270b151f93aa49a09a8c9a5fb6ddebb6/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:25.638814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.639149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.639425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.669645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] MFA Rules not processed for user `270b151f93aa49a09a8c9a5fb6ddebb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:25.670877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:25.736010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 934/1878] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:25.770586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:25.772999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:25.829753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324179da-bc53-4bfa-b4ce-79bc5cc280ae admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:25.833473 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f97328-135d-4044-a9bf-5826a91b165b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:25.834755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f97328-135d-4044-a9bf-5826a91b165b service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:25.834989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f97328-135d-4044-a9bf-5826a91b165b service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:25.835262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f97328-135d-4044-a9bf-5826a91b165b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:25.844482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f97328-135d-4044-a9bf-5826a91b165b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:25.845754 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f97328-135d-4044-a9bf-5826a91b165b service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:25.854948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 945/1879] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:27.762389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:28.137425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:28.138866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:28.203297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb1eea-07ad-4eaf-96ee-3b79c44afb70 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:28.207843 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:28.208510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:28.208787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:28.209660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] PATH_INFO: `/v3/users/87f912b9a1dd4aeb8f6b3b0ce398483c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:28.210769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f912b9a1dd4aeb8f6b3b0ce398483c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:28.212798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:28.367626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:28.417301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f912b9a1dd4aeb8f6b3b0ce398483c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:28.417301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f912b9a1dd4aeb8f6b3b0ce398483c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:28.417301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f912b9a1dd4aeb8f6b3b0ce398483c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:28.499265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f912b9a1dd4aeb8f6b3b0ce398483c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:28.508039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f912b9a1dd4aeb8f6b3b0ce398483c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:28.528007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 935/1880] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:28 2022] DELETE /identity/v3/users/87f912b9a1dd4aeb8f6b3b0ce398483c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:28.532527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:28.532815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:28.533095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:28.556234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:28.557446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:28.619610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 946/1881] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:28.629920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:28.631973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:28.694384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1e56-6d23-4ddc-95ea-f1227039ee78 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:28.698318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:28.698939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:28.699242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:28.699509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:28.709377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:28.710834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:28.721535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 936/1882] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:29.170082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:29.171625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:29.183104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:29.189793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6d91-75e5-49e3-a665-1de1aded91f6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:29.195609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:29.196179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:29.196480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:29.196761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] PATH_INFO: `/v3/projects/9ec611a4c22a48ce80e77b85cb0bea84` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:29.197834 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec611a4c22a48ce80e77b85cb0bea84)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:29.199849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:29.224399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:29.316034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec611a4c22a48ce80e77b85cb0bea84', 'request_id': 'req-8b6fe0ff-5396-42a7-819f-11663caba7ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.390863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec611a4c22a48ce80e77b85cb0bea84', 'request_id': 'req-8b6fe0ff-5396-42a7-819f-11663caba7ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.398013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec611a4c22a48ce80e77b85cb0bea84', 'request_id': 'req-8b6fe0ff-5396-42a7-819f-11663caba7ad'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.411153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] The token cache is being invalidate because project 9ec611a4c22a48ce80e77b85cb0bea84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:29.411443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.412202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.413742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 947/1883] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:29 2022] DELETE /identity/v3/projects/9ec611a4c22a48ce80e77b85cb0bea84 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:29.417626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:29.428683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84087a-a12b-4d8c-bc3d-5c95d0c01755 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:29.489211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:29.489835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:29.490153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:29.490422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] PATH_INFO: `/v3/users/84555ebf495149e198bf8c8abd687305` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:29.491528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=84555ebf495149e198bf8c8abd687305)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:29.493460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:29.678022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84555ebf495149e198bf8c8abd687305'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.678685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84555ebf495149e198bf8c8abd687305'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.679324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84555ebf495149e198bf8c8abd687305'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.801815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84555ebf495149e198bf8c8abd687305'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.811530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-192f0fa2-5816-4578-926a-1aff9be58dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84555ebf495149e198bf8c8abd687305'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:29.831491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 937/1884] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:29 2022] DELETE /identity/v3/users/84555ebf495149e198bf8c8abd687305 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:30.048604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:30.421859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:30.424178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:30.444019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fe0ff-5396-42a7-819f-11663caba7ad admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:30.496104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:30.529664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:30.530300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:30.530575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:30.530845 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] PATH_INFO: `/v3/projects/9f05c0e1acf7453ab4224960e2392391` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:30.531972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f05c0e1acf7453ab4224960e2392391)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:30.533933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:30.647706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f05c0e1acf7453ab4224960e2392391', 'request_id': 'req-9e2298b7-db10-4e3e-92e7-33194a888800'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:30.715946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f05c0e1acf7453ab4224960e2392391', 'request_id': 'req-9e2298b7-db10-4e3e-92e7-33194a888800'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:30.721637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f05c0e1acf7453ab4224960e2392391', 'request_id': 'req-9e2298b7-db10-4e3e-92e7-33194a888800'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:30.733179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] The token cache is being invalidate because project 9f05c0e1acf7453ab4224960e2392391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:30.733466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:30.734098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9e2298b7-db10-4e3e-92e7-33194a888800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:30.735581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 948/1885] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:30 2022] DELETE /identity/v3/projects/9f05c0e1acf7453ab4224960e2392391 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:30.740591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:30.740873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:30.741144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:30.760771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:30.761978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:30.823156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 938/1886] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:30.831108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:30.831439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:30.831715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:30.854982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:30.856237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:30.864486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 949/1887] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:30.868614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:30.878305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a517b4-051a-4f9c-b366-b3a98cdb1371 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:30.883404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:30.883998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:30.884316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:30.884589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:30.885233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:30.885534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:30.887112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:30.899575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 939/1888] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:30.903639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:30.913176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73834bb9-2f5d-467b-bb70-f7b41ebf426a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:30.917856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:30.918449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:30.918729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:30.918995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:30.919626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:30.921682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:31.055211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 950/1889] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:30 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:31.059322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:31.072934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b496c-c7c5-40df-b45a-3a09c3b7bc8a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:31.151686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:31.152340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.153077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.153077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.153526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:31.155467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:31.174083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:31.276233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 940/1890] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:31 2022] POST /identity/v3/users => generated 305 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:31.276360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:31.282864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acef38-0dee-4fe0-bd0c-61c471e07ca5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:31.284259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:31.293074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:31.293836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.294211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.294579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.295330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:31.297189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:31.310799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 951/1891] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:31.314968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:31.326819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30933992-6a84-41f3-bac7-18be90c3a3b4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:31.332217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595fae7-53c1-406a-a5ef-2bba649a3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:31.332846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595fae7-53c1-406a-a5ef-2bba649a3f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.333303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595fae7-53c1-406a-a5ef-2bba649a3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.334228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595fae7-53c1-406a-a5ef-2bba649a3f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.335038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595fae7-53c1-406a-a5ef-2bba649a3f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:31.337390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595fae7-53c1-406a-a5ef-2bba649a3f5f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:31.350280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 941/1892] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:31.354922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:31.366121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7b83-c39f-454e-be2b-6d4b95293f18 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:31.371352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:31.372199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.372483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.372756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] PATH_INFO: `/v3/projects/ab834403f1a94e5ab649032dc2bb80c8/users/110166dcfbac488296c26f4b125a0668/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.389024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab834403f1a94e5ab649032dc2bb80c8, user_id=110166dcfbac488296c26f4b125a0668, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:31.392365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:31.505559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 952/1893] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:31 2022] PUT /identity/v3/projects/ab834403f1a94e5ab649032dc2bb80c8/users/110166dcfbac488296c26f4b125a0668/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:31.517192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.518725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.519168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.557578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] MFA Rules not processed for user `110166dcfbac488296c26f4b125a0668`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:31.558961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:31.643486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 942/1894] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:31 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:31.660070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:31.736331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da22e89-d252-47ae-8a77-6ba176e854df admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:31.741536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:31.742363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.742697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.746600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.746600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:31.746600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:31.889701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 953/1895] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:31 2022] POST /identity/v3/projects => generated 409 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:31.894136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:31.905726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb4ed6-a24a-4b9e-ab32-f8bffb6d61b0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:31.961334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:31.961933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:31.962236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:31.962501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:31.963111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:31.965114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:32.036699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 943/1896] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:31 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:32.040872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:32.051027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5fbe7-bd44-4e4b-a647-9c307490849c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:32.056409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:32.057020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.057300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.057576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:32.058216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:32.059815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:32.071223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 954/1897] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:32.075439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:32.085680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59d35-5e70-4cf8-8a51-a5ba8ee22d1f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:32.090456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:32.091116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.091395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.091670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] PATH_INFO: `/v3/projects/34fccbcae7cb4c518d66e75036652b39/users/1d48644cf18f499b9f6bfe582c612fe9/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:32.099872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.106858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fccbcae7cb4c518d66e75036652b39, user_id=1d48644cf18f499b9f6bfe582c612fe9, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:32.109808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:32.206840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 944/1898] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:32 2022] PUT /identity/v3/projects/34fccbcae7cb4c518d66e75036652b39/users/1d48644cf18f499b9f6bfe582c612fe9/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:32.210744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:32.221070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bfb7-8aeb-4f63-9f82-49db40bd5a04 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:32.235725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.284526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4bdff-956b-4915-a5de-f2cb181a98e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:32.285058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bdff-956b-4915-a5de-f2cb181a98e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.285342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bdff-956b-4915-a5de-f2cb181a98e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.285611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bdff-956b-4915-a5de-f2cb181a98e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:32.286247 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4bdff-956b-4915-a5de-f2cb181a98e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:32.288039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4bdff-956b-4915-a5de-f2cb181a98e2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:32.298006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 955/1899] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:32.301806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:32.312778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58b4ae-b6cb-4ff2-b224-f81a0c5cb1e6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:32.317447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:32.318734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.319024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.319298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] PATH_INFO: `/v3/projects/34fccbcae7cb4c518d66e75036652b39/users/1d48644cf18f499b9f6bfe582c612fe9/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:32.321278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fccbcae7cb4c518d66e75036652b39, user_id=1d48644cf18f499b9f6bfe582c612fe9, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:32.322249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.323920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:32.334262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 945/1900] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:32 2022] PUT /identity/v3/projects/34fccbcae7cb4c518d66e75036652b39/users/1d48644cf18f499b9f6bfe582c612fe9/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:32.345164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.345454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.345734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:32.373839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] MFA Rules not processed for user `1d48644cf18f499b9f6bfe582c612fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:32.375054 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:32.390809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.442127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 956/1901] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:32 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:32.452892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:32.454575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:32.482388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.517749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73837b-4046-421e-8d39-3d5e2dece028 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:32.521654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:32.522411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.522695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.522958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:32.533448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:32.535195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:32.546543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 946/1902] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:32 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:32.550948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.654386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.842598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.969883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:32.971814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:32.986189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:32.987619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6bd27-d802-4a91-a33a-81bd25c77e21 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:32.992467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6dd82-51d2-43af-9bab-4c41381e3fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:32.993166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6dd82-51d2-43af-9bab-4c41381e3fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:32.993505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6dd82-51d2-43af-9bab-4c41381e3fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:32.993960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6dd82-51d2-43af-9bab-4c41381e3fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:33.057156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6dd82-51d2-43af-9bab-4c41381e3fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:33.058607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6dd82-51d2-43af-9bab-4c41381e3fe9 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:33.067354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 957/1903] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:32 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:33.098945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:33.602301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:36.270542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:36.272894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:36.335647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:36.357576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e244a-0ae2-457a-8f17-2a7a183419ec service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:36.363383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:36.364869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:36.365397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:36.365831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] PATH_INFO: `/v3/users/443c4d399e994a4ea5cbb50155bfd535` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:36.367590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] RBAC: Authorizing `identity:delete_user(user_id=443c4d399e994a4ea5cbb50155bfd535)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:36.369077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:36.630301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443c4d399e994a4ea5cbb50155bfd535'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:36.635047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443c4d399e994a4ea5cbb50155bfd535'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:36.637383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443c4d399e994a4ea5cbb50155bfd535'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:36.704788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443c4d399e994a4ea5cbb50155bfd535'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:36.731978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443c4d399e994a4ea5cbb50155bfd535'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:36.756412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 947/1904] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:36 2022] DELETE /identity/v3/users/443c4d399e994a4ea5cbb50155bfd535 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:36.761616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:36.762426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:36.762703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:36.783155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:36.784753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:36.891796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 958/1905] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:36.900633 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:36.902188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:36.978078 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33bf41-41d5-44c5-970a-15758761026a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:36.982394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5aa65-c71c-4ff1-83f4-9f374f7eb9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:36.983094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5aa65-c71c-4ff1-83f4-9f374f7eb9fe service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:36.983364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5aa65-c71c-4ff1-83f4-9f374f7eb9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:36.983620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5aa65-c71c-4ff1-83f4-9f374f7eb9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:36.995468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5aa65-c71c-4ff1-83f4-9f374f7eb9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:36.996946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5aa65-c71c-4ff1-83f4-9f374f7eb9fe service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:37.008779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 948/1906] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:37.039946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:37.041562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:37.054881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6d6a5-4a12-409a-9ecc-bc6197d2eea1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:37.060707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:37.061349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:37.061626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:37.061897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] PATH_INFO: `/v3/users/514ba228bc314c18ad2f3bc3bc5c78ca` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:37.063110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=514ba228bc314c18ad2f3bc3bc5c78ca)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:37.065111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:37.066828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:37.321734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514ba228bc314c18ad2f3bc3bc5c78ca'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.379142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514ba228bc314c18ad2f3bc3bc5c78ca'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.390708 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514ba228bc314c18ad2f3bc3bc5c78ca'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.392404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514ba228bc314c18ad2f3bc3bc5c78ca'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.392902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514ba228bc314c18ad2f3bc3bc5c78ca'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.422529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 959/1907] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:37 2022] DELETE /identity/v3/users/514ba228bc314c18ad2f3bc3bc5c78ca => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:37.427035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:37.427881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:37.428173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:37.448960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:37.450182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:37.488865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:37.490849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:37.550351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 949/1908] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:37.561362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:37.562934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:37.570788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:37.584575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29356617-a435-40ef-9faa-d5a26aee7334 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:37.591194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:37.591839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:37.592333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:37.592612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] PATH_INFO: `/v3/projects/e0e51503a67f430ba9c7d27d3e52975d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:37.593916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e51503a67f430ba9c7d27d3e52975d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:37.595871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:37.637380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e38a9-97a3-48a2-9761-885bd6aeaaa3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:37.641040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:37.641916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:37.642196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:37.642526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:37.652424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:37.654044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:37.666159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 950/1909] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:37.711811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e51503a67f430ba9c7d27d3e52975d', 'request_id': 'req-20791a90-11aa-4289-b90f-32949d2248e3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.807626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e51503a67f430ba9c7d27d3e52975d', 'request_id': 'req-20791a90-11aa-4289-b90f-32949d2248e3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.814202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e51503a67f430ba9c7d27d3e52975d', 'request_id': 'req-20791a90-11aa-4289-b90f-32949d2248e3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.825593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] The token cache is being invalidate because project e0e51503a67f430ba9c7d27d3e52975d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:37.825882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.826543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:37.828081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 960/1910] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:37 2022] DELETE /identity/v3/projects/e0e51503a67f430ba9c7d27d3e52975d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:37.832324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:37.842525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7d4d4-a4e6-4ca1-bf98-97ee661e7352 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:37.917493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:37.919061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:37.919438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:37.919808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] PATH_INFO: `/v3/users/15c62c6bf2c24611a8fa3e76777f96b6` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:37.921364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c62c6bf2c24611a8fa3e76777f96b6)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:37.923792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:38.086271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c62c6bf2c24611a8fa3e76777f96b6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.087044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c62c6bf2c24611a8fa3e76777f96b6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.088658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c62c6bf2c24611a8fa3e76777f96b6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.136084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:38.137638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:38.152232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20791a90-11aa-4289-b90f-32949d2248e3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:38.156404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c62c6bf2c24611a8fa3e76777f96b6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.157228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:38.157859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:38.158257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:38.158541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] PATH_INFO: `/v3/projects/a4440e890c4a498d8ae85b2f2eefde6b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:38.159667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4440e890c4a498d8ae85b2f2eefde6b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:38.161652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:38.164840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:38.166893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c62c6bf2c24611a8fa3e76777f96b6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.195215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 951/1911] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:37 2022] DELETE /identity/v3/users/15c62c6bf2c24611a8fa3e76777f96b6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:38.311800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4440e890c4a498d8ae85b2f2eefde6b', 'request_id': 'req-23756444-9d3e-457c-a01a-e75bab8345d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.373668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4440e890c4a498d8ae85b2f2eefde6b', 'request_id': 'req-23756444-9d3e-457c-a01a-e75bab8345d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.380435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4440e890c4a498d8ae85b2f2eefde6b', 'request_id': 'req-23756444-9d3e-457c-a01a-e75bab8345d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.393093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] The token cache is being invalidate because project a4440e890c4a498d8ae85b2f2eefde6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:38.393474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.394179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.396176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 961/1912] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:38 2022] DELETE /identity/v3/projects/a4440e890c4a498d8ae85b2f2eefde6b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:38.406473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:38.406473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:38.445538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:38.483668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b5636-8d99-4fa1-a22e-66b093f08a23 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:38.549982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:38.550744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:38.551099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:38.551441 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] PATH_INFO: `/v3/users/7dac083b3ce6441da8a31bcbfe31b661` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:38.553347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dac083b3ce6441da8a31bcbfe31b661)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:38.555445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:38.772583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:38.774133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:38.789399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23756444-9d3e-457c-a01a-e75bab8345d5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:38.794175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:38.795042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:38.795322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:38.795618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] PATH_INFO: `/v3/projects/bf64ba8b91e94fd09fc6b42026ede7d1` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:38.796781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf64ba8b91e94fd09fc6b42026ede7d1)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:38.799588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:38.873325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dac083b3ce6441da8a31bcbfe31b661'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.874020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dac083b3ce6441da8a31bcbfe31b661'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.874639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dac083b3ce6441da8a31bcbfe31b661'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.950664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dac083b3ce6441da8a31bcbfe31b661'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.958237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-57ead32b-04c2-404c-bc22-db0bf3caeaa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dac083b3ce6441da8a31bcbfe31b661'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:38.975972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 952/1913] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:38 2022] DELETE /identity/v3/users/7dac083b3ce6441da8a31bcbfe31b661 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:38.997725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf64ba8b91e94fd09fc6b42026ede7d1', 'request_id': 'req-172697e9-8157-40b5-858d-86f38007f9e2'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.099300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf64ba8b91e94fd09fc6b42026ede7d1', 'request_id': 'req-172697e9-8157-40b5-858d-86f38007f9e2'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.105677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf64ba8b91e94fd09fc6b42026ede7d1', 'request_id': 'req-172697e9-8157-40b5-858d-86f38007f9e2'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.118569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] The token cache is being invalidate because project bf64ba8b91e94fd09fc6b42026ede7d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:39.118854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.119491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-172697e9-8157-40b5-858d-86f38007f9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.121066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 962/1914] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:38 2022] DELETE /identity/v3/projects/bf64ba8b91e94fd09fc6b42026ede7d1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:39.125971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.132947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.132947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.149935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:39.152308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.202449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:39.214681 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 953/1915] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:39.222656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.223040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.223491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.244830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:39.246208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.254844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 963/1916] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:39.259283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.269291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66decd6-4584-4527-8356-ba6d37915aa7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.274010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.274712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.275062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.275435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.276169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:39.276489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.279876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.295397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 954/1917] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:39.299898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.311146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a48383-fa26-4918-9f0b-27504e7ed4f4 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.315781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.316427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.316705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.316973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.317601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.319688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.406438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.408774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.425240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 964/1918] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:39.429647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.448220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65750a-f732-4f1e-ab95-3f3b515b747e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.494726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:39.508548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df5ec0-9454-4e6c-9bd9-a3e0c0c430c6 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.513465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.514079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.514358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.514816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.515546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.517778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.518572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.519214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.523097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.523097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] PATH_INFO: `/v3/projects/9277ea301c0748748b55ad5e766d130b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.523097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9277ea301c0748748b55ad5e766d130b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.523589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.596866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 965/1919] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/users => generated 282 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:39.601578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.611047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef7b37-296c-4f77-9816-7825cf314517 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.615771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.616394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.616673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.616945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.617577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.619297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.629695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 966/1920] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:39.633554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.640552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9277ea301c0748748b55ad5e766d130b', 'request_id': 'req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.643313 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c35fe-ce96-4343-ad4c-3e3058cfa971 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.699383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9277ea301c0748748b55ad5e766d130b', 'request_id': 'req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.703554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.704214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.704707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.705055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.705791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.710843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.710965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9277ea301c0748748b55ad5e766d130b', 'request_id': 'req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.719352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 967/1921] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:39.722889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] The token cache is being invalidate because project 9277ea301c0748748b55ad5e766d130b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:39.723420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.723555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.725353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1dfa0d1c-ab6d-4f34-9835-424263cd5f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:39.730922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 955/1922] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:39 2022] DELETE /identity/v3/projects/9277ea301c0748748b55ad5e766d130b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:39.732606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db9f8-43a2-423f-a239-294e9dfde6d6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.734283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db9f8-43a2-423f-a239-294e9dfde6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.734283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db9f8-43a2-423f-a239-294e9dfde6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.734476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb414b-05ae-4885-82e4-85322eeabb94 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.739308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e909b-1250-4db9-8e52-2015421487bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.739999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e909b-1250-4db9-8e52-2015421487bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.740322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e909b-1250-4db9-8e52-2015421487bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.740598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e909b-1250-4db9-8e52-2015421487bc admin admin] PATH_INFO: `/v3/projects/2730501ba82d4c54be8fcccc2d597c7d/users/fd4c0f6a4eda4a4a9d3bea9b5ac4610d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.756856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e909b-1250-4db9-8e52-2015421487bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2730501ba82d4c54be8fcccc2d597c7d, user_id=fd4c0f6a4eda4a4a9d3bea9b5ac4610d, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.759825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e909b-1250-4db9-8e52-2015421487bc admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.766667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d21db9f8-43a2-423f-a239-294e9dfde6d6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:39.768102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d21db9f8-43a2-423f-a239-294e9dfde6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.784088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 956/1923] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:39.788073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.788246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.788538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.808405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:39.809636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.818153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 957/1924] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:39.821991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.829084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 968/1925] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:39 2022] PUT /identity/v3/projects/2730501ba82d4c54be8fcccc2d597c7d/users/fd4c0f6a4eda4a4a9d3bea9b5ac4610d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:39.836591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8761d-eb69-4b34-9833-4205142044e1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.840620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.841031 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.841204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.869071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] MFA Rules not processed for user `fd4c0f6a4eda4a4a9d3bea9b5ac4610d`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:39.870284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.893190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:39.899988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.900393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.900675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.900952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.903510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:39.903510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.903677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.916193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 958/1926] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:46:39.920222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.930356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde5527-d680-47a9-aaed-4ac90fdebc04 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.937896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.937896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.937896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.937896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.938362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.939937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:39.943522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 969/1927] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:39.949538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:39.951697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:39.971197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e975c-a626-437b-b29f-b277f9ac173d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:39.977033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36030925-f856-4041-8a61-627f155b310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:39.977931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36030925-f856-4041-8a61-627f155b310a admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:39.977931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36030925-f856-4041-8a61-627f155b310a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:39.978242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36030925-f856-4041-8a61-627f155b310a admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:39.978843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36030925-f856-4041-8a61-627f155b310a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:39.981117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36030925-f856-4041-8a61-627f155b310a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.017701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 959/1928] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:40.022442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.066552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 970/1929] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:39 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:40.072166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36030925-f856-4041-8a61-627f155b310a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.092685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:40.165772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26d4f0-0e6e-46e7-b01b-0b7109b51296 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.255131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36030925-f856-4041-8a61-627f155b310a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.273862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.274756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.275149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.275539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.276321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.280685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.310291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.311183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.311468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.311794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.313082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.315299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.362483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 971/1930] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:40 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:40.367248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.390130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6926cf0-eca9-4c9b-a5e6-032018144023 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.405183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.405183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.405183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.405183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.405731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.405731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.421551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 972/1931] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:40.428988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.435215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 960/1932] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:40 2022] POST /identity/v3/users => generated 285 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:40.436851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.450449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef217f9-7eb1-4790-bf03-c0beaf5a77bb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.455317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210c880-c7f1-476c-b528-2dd936bfaa44 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.456643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.457442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.457874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.458252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.459181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.460974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.463042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.474843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.474843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.475110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] PATH_INFO: `/v3/projects/d1f990daabf24426998af05b1a17a8c1/users/65584c1496624848b20f2c64ef88f3ed/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.483496 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 961/1933] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:40.489021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.522838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ba4c2-1551-415f-85fe-92296b54a57d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.529708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.530150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f990daabf24426998af05b1a17a8c1, user_id=65584c1496624848b20f2c64ef88f3ed, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.530395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.537571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.537571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.537571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.537571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.537908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.554226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 962/1934] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:40.556453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.575760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7adf26-f659-4dbe-bd67-83fcc9c5e2ed admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.575760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c50b01-28a3-4769-9b2f-0b85db7d9190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.577132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c50b01-28a3-4769-9b2f-0b85db7d9190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.577586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c50b01-28a3-4769-9b2f-0b85db7d9190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.578207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c50b01-28a3-4769-9b2f-0b85db7d9190 admin admin] PATH_INFO: `/v3/projects/52daecc0758a40dfaa0ae239a6481ca9/users/9a284ea10de0453eb18b11dcb7267592/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.601075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c50b01-28a3-4769-9b2f-0b85db7d9190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52daecc0758a40dfaa0ae239a6481ca9, user_id=9a284ea10de0453eb18b11dcb7267592, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.605308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c50b01-28a3-4769-9b2f-0b85db7d9190 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:40.668489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 973/1935] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:40 2022] PUT /identity/v3/projects/d1f990daabf24426998af05b1a17a8c1/users/65584c1496624848b20f2c64ef88f3ed/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:40.672575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.709673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 963/1936] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:40 2022] PUT /identity/v3/projects/52daecc0758a40dfaa0ae239a6481ca9/users/9a284ea10de0453eb18b11dcb7267592/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:40.720527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.720818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.721092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.767440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] MFA Rules not processed for user `9a284ea10de0453eb18b11dcb7267592`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:40.769244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:40.800770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:40.807008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a9be7-1d29-441d-85e4-92253da662ea admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:40.901366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 964/1937] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:40.905762 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:40.959580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:40.960288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:40.960576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:40.961190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:40.961876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:40.977084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.031085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 974/1938] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:41.032365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.039018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:41.046739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95616c2a-085c-4ada-83d0-ceb327d12c04 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.051160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034490a1-0347-4c2f-8286-0989f92bc151 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.053613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b95433-3229-4c5a-83c0-55abe4e2c123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.054594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b95433-3229-4c5a-83c0-55abe4e2c123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.057500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b95433-3229-4c5a-83c0-55abe4e2c123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.057500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b95433-3229-4c5a-83c0-55abe4e2c123 admin admin] PATH_INFO: `/v3/projects/d1f990daabf24426998af05b1a17a8c1/users/65584c1496624848b20f2c64ef88f3ed/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.057698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.057922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.058216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.058493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.058963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b95433-3229-4c5a-83c0-55abe4e2c123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f990daabf24426998af05b1a17a8c1, user_id=65584c1496624848b20f2c64ef88f3ed, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.059291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.061458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.062560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b95433-3229-4c5a-83c0-55abe4e2c123 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.088874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 975/1939] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:41 2022] PUT /identity/v3/projects/d1f990daabf24426998af05b1a17a8c1/users/65584c1496624848b20f2c64ef88f3ed/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:41.090734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.091119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.091632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.128543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] MFA Rules not processed for user `65584c1496624848b20f2c64ef88f3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:41.130396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:41.204282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 965/1940] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:40 2022] POST /identity/v3/projects => generated 369 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:41.206053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.245639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 976/1941] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:41.297433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.299019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:41.315820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20172b3f-dc52-4d64-a352-5e595c27034e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.332362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.334294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.336868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.337586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.340187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.342421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.386249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d1973-2d0e-4f19-89c6-a0597434289e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.447510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.448449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.448806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.449291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.479721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 966/1942] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:41 2022] POST /identity/v3/users => generated 291 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:41.484272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.508944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440898ab-5dcf-4308-ac29-4441531a1e95 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.515982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.517339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.517847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.519063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.520113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.522319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.555272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 967/1943] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:41.556342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.557355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:41.568522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879863a3-1ffa-4906-b887-569f4e422384 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.574263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.575062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.576081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.576081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] PATH_INFO: `/v3/projects/9490ebca87b34fd0b268a08722388432/users/3d845cbce19545c1996b14c71c86dae5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.592557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.593129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.593811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:41.599830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9490ebca87b34fd0b268a08722388432, user_id=3d845cbce19545c1996b14c71c86dae5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.602580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.604374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 977/1944] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:41.689694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 968/1945] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:41 2022] PUT /identity/v3/projects/9490ebca87b34fd0b268a08722388432/users/3d845cbce19545c1996b14c71c86dae5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:41.690703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.706100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f0ea4-b5db-497d-9168-aafd5e686200 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.773997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b185bb-156c-45a1-be63-cb28806a846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.774612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b185bb-156c-45a1-be63-cb28806a846c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.774894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b185bb-156c-45a1-be63-cb28806a846c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.775164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b185bb-156c-45a1-be63-cb28806a846c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.775815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b185bb-156c-45a1-be63-cb28806a846c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.777500 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b185bb-156c-45a1-be63-cb28806a846c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.788384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 978/1946] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:41.795250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.805504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a19cf0-988a-4160-8897-7c3b438c56b6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:41.810198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:41.810853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.811129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.811399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] PATH_INFO: `/v3/projects/9490ebca87b34fd0b268a08722388432/users/3d845cbce19545c1996b14c71c86dae5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.813410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9490ebca87b34fd0b268a08722388432, user_id=3d845cbce19545c1996b14c71c86dae5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:41.817468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:41.826609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 969/1947] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:41 2022] PUT /identity/v3/projects/9490ebca87b34fd0b268a08722388432/users/3d845cbce19545c1996b14c71c86dae5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:41.838692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:41.838982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:41.839265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:41.874216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] MFA Rules not processed for user `3d845cbce19545c1996b14c71c86dae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:41.875531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:41.937445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:41.980615 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 979/1948] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:42.008123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a7aa-556b-4141-9ee4-5636c29dd4c5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:42.011870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:42.012552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:42.012830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:42.013096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:42.074349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:42.075797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:42.088944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 970/1949] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:42.090592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:42.093188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:42.093998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:42.095597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:42.108674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d9bb-9c28-4ad6-8a51-f777e1013bde service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:42.113359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:42.114120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:42.114563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:42.115560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:42.126145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:42.127682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:42.140536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 971/1950] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:42.182709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374029-96de-423c-8fa6-4baeb947bbb1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:42.187504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:42.188262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:42.188498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:42.188764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] PATH_INFO: `/v3/users/270b151f93aa49a09a8c9a5fb6ddebb6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:42.189907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] RBAC: Authorizing `identity:delete_user(user_id=270b151f93aa49a09a8c9a5fb6ddebb6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:42.191881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:42.195132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:42.275242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:42.284243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce5fb5-a34a-4916-a7d7-52a8870702e2 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:42.288669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5d2a9-218b-4466-a515-f2984d8eaa6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:42.290581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d2a9-218b-4466-a515-f2984d8eaa6d service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:42.290581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d2a9-218b-4466-a515-f2984d8eaa6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:42.290581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d2a9-218b-4466-a515-f2984d8eaa6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:42.354053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5d2a9-218b-4466-a515-f2984d8eaa6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:42.355561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5d2a9-218b-4466-a515-f2984d8eaa6d service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:42.365387 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 972/1951] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:42.406491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270b151f93aa49a09a8c9a5fb6ddebb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:42.499189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270b151f93aa49a09a8c9a5fb6ddebb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:42.508538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270b151f93aa49a09a8c9a5fb6ddebb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:42.509250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270b151f93aa49a09a8c9a5fb6ddebb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:42.509866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270b151f93aa49a09a8c9a5fb6ddebb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:42.537340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 980/1952] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:42 2022] DELETE /identity/v3/users/270b151f93aa49a09a8c9a5fb6ddebb6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:42.541936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:42.541936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:42.542452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:42.584215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:42.592613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:42.636339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:42.730993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 973/1953] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:42.742229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:42.743809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:42.855466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c338f-998b-490e-b3c9-6a453ea8945a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:42.866641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd28a5c-4b19-4222-bbcf-fd2bb10dbeeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:42.867523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28a5c-4b19-4222-bbcf-fd2bb10dbeeb service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:42.868046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28a5c-4b19-4222-bbcf-fd2bb10dbeeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:42.868560 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28a5c-4b19-4222-bbcf-fd2bb10dbeeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:42.880012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd28a5c-4b19-4222-bbcf-fd2bb10dbeeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:42.882105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd28a5c-4b19-4222-bbcf-fd2bb10dbeeb service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:42.892784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 981/1954] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:43.371241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:43.371672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:43.387663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60069d5d-7e2d-4627-b0c1-9a7602f52531 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:43.396998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:43.397674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:43.400118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:43.400118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] PATH_INFO: `/v3/projects/eccd05d89a0949b48f3ab118b66e86ab` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:43.400118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccd05d89a0949b48f3ab118b66e86ab)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:43.401759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:43.541983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccd05d89a0949b48f3ab118b66e86ab', 'request_id': 'req-1681452c-303f-42b7-ac89-e106beb74ea1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:43.607463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccd05d89a0949b48f3ab118b66e86ab', 'request_id': 'req-1681452c-303f-42b7-ac89-e106beb74ea1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:43.614350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccd05d89a0949b48f3ab118b66e86ab', 'request_id': 'req-1681452c-303f-42b7-ac89-e106beb74ea1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:43.627389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] The token cache is being invalidate because project eccd05d89a0949b48f3ab118b66e86ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:43.627587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:43.628277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1681452c-303f-42b7-ac89-e106beb74ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:43.629813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 974/1955] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:43 2022] DELETE /identity/v3/projects/eccd05d89a0949b48f3ab118b66e86ab => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:43.635354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:43.640955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:43.640955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:43.657600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:43.658817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:43.732565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 982/1956] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:43.740956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:43.741277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:43.741514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:43.763030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:43.764300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:43.772789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 975/1957] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:43.777258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:43.789331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a3a4a-339d-4680-b355-31fc641fdeb9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:43.794517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:43.795357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:43.795647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:43.795921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:43.796668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:43.796977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:43.798638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:43.815301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 983/1958] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:43.817727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:43.827577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f043f0-0571-4948-8cc4-c63d83f8f44d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:43.833382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:43.834460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:43.834742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:43.835013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:43.835652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:43.838412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.015544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 976/1959] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:43 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:44.017766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:44.028583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90051-95b7-4c5a-b74d-9693a2f52575 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:44.038676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:44.097334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:44.103934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.103934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.103934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.103934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:44.103934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.203835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 984/1960] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:44 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:44.210341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:44.221646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f5f-d2cd-4af5-b465-d0815a1b49d9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:44.229216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:44.230021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.230361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.230696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.231447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:44.233202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.247831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 977/1961] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:44.250048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:44.261387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edefdfa-2b9e-4b3f-9d42-fa22f03f30ee admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:44.268940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff77fca-ff18-457c-8e38-1d2021fd0527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:44.269539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff77fca-ff18-457c-8e38-1d2021fd0527 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.269819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff77fca-ff18-457c-8e38-1d2021fd0527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.271849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff77fca-ff18-457c-8e38-1d2021fd0527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.271849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff77fca-ff18-457c-8e38-1d2021fd0527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:44.273733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff77fca-ff18-457c-8e38-1d2021fd0527 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.288002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 985/1962] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:44.292107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:44.303897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe716f02-3c4a-4da2-902f-95826b962a64 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:44.308824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:44.309510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.316047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.316047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] PATH_INFO: `/v3/projects/dd3cb76a9a344c7bb8af35f93465c3ee/users/62e34efd21e74468b53a267f28364dee/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.326543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3cb76a9a344c7bb8af35f93465c3ee, user_id=62e34efd21e74468b53a267f28364dee, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:44.329235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.432187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 978/1963] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:44 2022] PUT /identity/v3/projects/dd3cb76a9a344c7bb8af35f93465c3ee/users/62e34efd21e74468b53a267f28364dee/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:44.437999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.438291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.438565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.470889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] MFA Rules not processed for user `62e34efd21e74468b53a267f28364dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:44.472495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:44.546887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 986/1964] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:44.560387 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:44.562183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:44.603269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:44.621074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023571e-9dac-46f5-91c2-88a12ab3ba99 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:44.624941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd64da7-8283-4484-aaa5-68cf1540557d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:44.625562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd64da7-8283-4484-aaa5-68cf1540557d service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.625871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd64da7-8283-4484-aaa5-68cf1540557d service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.626149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd64da7-8283-4484-aaa5-68cf1540557d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.634516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd64da7-8283-4484-aaa5-68cf1540557d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:44.635974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd64da7-8283-4484-aaa5-68cf1540557d service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.645086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 979/1965] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:44.810789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:44.812324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:44.881543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e497-ff13-4662-839e-9b4671b071bd None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:44.886112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:44.886729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:44.887012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:44.887290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] PATH_INFO: `/v3/users/110166dcfbac488296c26f4b125a0668` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:44.888503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110166dcfbac488296c26f4b125a0668)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:44.890449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:44.915048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:45.114050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110166dcfbac488296c26f4b125a0668'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.178345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110166dcfbac488296c26f4b125a0668'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.190171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110166dcfbac488296c26f4b125a0668'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.191198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110166dcfbac488296c26f4b125a0668'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.192138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110166dcfbac488296c26f4b125a0668'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.217904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 987/1966] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:44 2022] DELETE /identity/v3/users/110166dcfbac488296c26f4b125a0668 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:45.233890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:45.234180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:45.234446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:45.254833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:45.256616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:45.326367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 980/1967] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:45.334034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:45.335579 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:45.395791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef69e2b-1f23-4749-9e41-db5ebf6da452 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:45.399949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:45.400670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:45.400964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:45.401282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:45.410226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:45.411681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:45.421367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 988/1968] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:45.809542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:45.811012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:45.824259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9024d5-9ccd-4479-874a-38eb41fb1ec3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:45.829713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:45.831010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:45.831010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:45.831010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] PATH_INFO: `/v3/projects/ab834403f1a94e5ab649032dc2bb80c8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:45.832138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab834403f1a94e5ab649032dc2bb80c8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:45.834304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:45.934963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab834403f1a94e5ab649032dc2bb80c8', 'request_id': 'req-9f826a79-5329-492a-8e92-dfb69b41df83'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.984797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab834403f1a94e5ab649032dc2bb80c8', 'request_id': 'req-9f826a79-5329-492a-8e92-dfb69b41df83'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:45.991701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab834403f1a94e5ab649032dc2bb80c8', 'request_id': 'req-9f826a79-5329-492a-8e92-dfb69b41df83'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.003027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] The token cache is being invalidate because project ab834403f1a94e5ab649032dc2bb80c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:46.003344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.003975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.005527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 981/1969] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:45 2022] DELETE /identity/v3/projects/ab834403f1a94e5ab649032dc2bb80c8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:46.009605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:46.019760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ec864-77b7-4961-8ee4-792c645c4a51 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:46.077955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:46.078598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:46.078877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:46.079148 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] PATH_INFO: `/v3/users/1d48644cf18f499b9f6bfe582c612fe9` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:46.080348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d48644cf18f499b9f6bfe582c612fe9)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:46.082305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:46.287645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:46.290176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d48644cf18f499b9f6bfe582c612fe9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.414485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d48644cf18f499b9f6bfe582c612fe9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.414485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d48644cf18f499b9f6bfe582c612fe9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.414485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d48644cf18f499b9f6bfe582c612fe9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.414949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ec8eb9cd-6b61-4234-893e-35c4837477d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d48644cf18f499b9f6bfe582c612fe9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:46.438389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 989/1970] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:46 2022] DELETE /identity/v3/users/1d48644cf18f499b9f6bfe582c612fe9 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:46.930196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:46.935288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:46.949064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f826a79-5329-492a-8e92-dfb69b41df83 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.015429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.015991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.016309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.016839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] PATH_INFO: `/v3/projects/34fccbcae7cb4c518d66e75036652b39` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.017958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34fccbcae7cb4c518d66e75036652b39)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.019856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.022731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:47.144544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34fccbcae7cb4c518d66e75036652b39', 'request_id': 'req-daed32e9-2ccf-4807-9455-335e1a81ec65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:47.221944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34fccbcae7cb4c518d66e75036652b39', 'request_id': 'req-daed32e9-2ccf-4807-9455-335e1a81ec65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:47.230837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34fccbcae7cb4c518d66e75036652b39', 'request_id': 'req-daed32e9-2ccf-4807-9455-335e1a81ec65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:47.243327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] The token cache is being invalidate because project 34fccbcae7cb4c518d66e75036652b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:47.243678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:47.244790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-daed32e9-2ccf-4807-9455-335e1a81ec65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:47.247104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 982/1971] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:46 2022] DELETE /identity/v3/projects/34fccbcae7cb4c518d66e75036652b39 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:47.253364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.253364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.253883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.275836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:47.277259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:47.339657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 990/1972] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:47.347495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.347779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.348050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.368714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:47.369919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:47.381547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 983/1973] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:47.382603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:47.392486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13e977-f989-4108-a071-b5df7c0c7241 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.397695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.398299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.398577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.398846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.399495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:47.399793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.401426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.413141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 991/1974] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:47.429211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:47.442402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b8bf1-74c1-473e-b8a0-0af5cf7aa6fb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.450346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.451327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.451683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.452220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.453706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.455929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.554062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 984/1975] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:47 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:47.558180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:47.569213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bfced-5617-4123-a723-4c161d234efa admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.633636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.634243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.634523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.634794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.635430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.637427 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.721988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 992/1976] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:47 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:47.727002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:47.737438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dfbb7-1d9f-4b1d-8918-a9b81d43a814 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.742578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.743185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.747021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.747021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.747021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.747021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.756828 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 985/1977] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:47.760879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:47.771652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f66067-f765-49c8-b24c-f59e26d2356a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.776237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07310b85-7c4a-40ce-a809-815c12adabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.776836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07310b85-7c4a-40ce-a809-815c12adabcc admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.777191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07310b85-7c4a-40ce-a809-815c12adabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.777414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07310b85-7c4a-40ce-a809-815c12adabcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.778076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07310b85-7c4a-40ce-a809-815c12adabcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.779661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07310b85-7c4a-40ce-a809-815c12adabcc admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.790629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 993/1978] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:47.794853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:47.805858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985c244-71c6-4951-9cf8-1113abaf6483 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:47.810629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:47.811316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.811593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.811863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] PATH_INFO: `/v3/projects/9b615ab48a5b4e31824a7e508790fd48/users/e2072a3d177043baaaa7a90eb8be7350/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.827362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b615ab48a5b4e31824a7e508790fd48, user_id=e2072a3d177043baaaa7a90eb8be7350, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:47.830634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:47.942299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 986/1979] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:47 2022] PUT /identity/v3/projects/9b615ab48a5b4e31824a7e508790fd48/users/e2072a3d177043baaaa7a90eb8be7350/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:47.956214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:47.956392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:47.956670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:47.993410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] MFA Rules not processed for user `e2072a3d177043baaaa7a90eb8be7350`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:47.994750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:48.012344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:48.063145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.064860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:48.065178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 994/1980] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:48.070471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.134553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc1859-149d-4eae-9d78-37c2a7d2e149 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.138487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516806e-628a-42c1-abe2-6752a6c82be1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.139887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.140574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.140852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.141126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.142022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.143945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.144188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.149617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.149617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.149617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] PATH_INFO: `/v3/users/9a284ea10de0453eb18b11dcb7267592` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.149617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a284ea10de0453eb18b11dcb7267592)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.149617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.210038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 995/1981] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:48 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:48.212995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.277695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:48.289729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d789ba-95e1-4067-9301-4225d8c131be admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.293641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a284ea10de0453eb18b11dcb7267592'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:48.294375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a284ea10de0453eb18b11dcb7267592'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:48.294501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.294998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a284ea10de0453eb18b11dcb7267592'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:48.295107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.295322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.295602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.296271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.298918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.401852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a284ea10de0453eb18b11dcb7267592'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:48.409435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5ccfcf4-46a9-4098-b180-2b8dc025b39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a284ea10de0453eb18b11dcb7267592'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:48.419844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 996/1982] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:48 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:48.423866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.425575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:48.430085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 987/1983] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:48 2022] DELETE /identity/v3/users/9a284ea10de0453eb18b11dcb7267592 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:48.434229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.434229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.434229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.446332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902ade7-9cd3-4643-9039-46c531409818 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.456408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:48.457724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:48.473028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:48.507491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.508139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.508455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.508727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.509373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.510983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.522314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 997/1984] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:48.524504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 988/1985] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:48.526915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.538085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a6370-a3a0-46c7-b47a-a711a13b51bc admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.541344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.542779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.542910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:48.543425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.543860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.544092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] PATH_INFO: `/v3/projects/6939a492744846cdbacb4e5c3b19c9ee/users/ea39ec2168ef4a5596d032b6ac513cfc/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.559918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6939a492744846cdbacb4e5c3b19c9ee, user_id=ea39ec2168ef4a5596d032b6ac513cfc, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.563995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.615280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf58d3-72db-40ea-b271-d9ce748d1f63 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.624954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.625650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.625926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.627024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.640357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.642245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.653797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 989/1986] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:48.682769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 998/1987] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:48 2022] PUT /identity/v3/projects/6939a492744846cdbacb4e5c3b19c9ee/users/ea39ec2168ef4a5596d032b6ac513cfc/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:48.685750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.699794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df678c-630a-4416-ac90-a21835c62ae2 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.777482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:48.841975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93c74a-c4df-4553-b609-e94c8eafe5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.841975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93c74a-c4df-4553-b609-e94c8eafe5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.841975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93c74a-c4df-4553-b609-e94c8eafe5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.841975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93c74a-c4df-4553-b609-e94c8eafe5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.843065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93c74a-c4df-4553-b609-e94c8eafe5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.843065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93c74a-c4df-4553-b609-e94c8eafe5a1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.843065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 990/1988] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:48.844467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:48.858980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2b0c5-1722-492f-afb3-a4eb15a2771e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:48.864230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:48.865475 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.865763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.866067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] PATH_INFO: `/v3/projects/6939a492744846cdbacb4e5c3b19c9ee/users/ea39ec2168ef4a5596d032b6ac513cfc/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.868048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6939a492744846cdbacb4e5c3b19c9ee, user_id=ea39ec2168ef4a5596d032b6ac513cfc, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:48.871264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:48.881901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 999/1989] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:48 2022] PUT /identity/v3/projects/6939a492744846cdbacb4e5c3b19c9ee/users/ea39ec2168ef4a5596d032b6ac513cfc/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:48.892199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:48.892486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:48.892783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:48.926375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] MFA Rules not processed for user `ea39ec2168ef4a5596d032b6ac513cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:48.928329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:49.008475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 991/1990] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:49.020995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:49.022975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:49.092631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07369f-866d-4812-9eab-a3f8991e6c2f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:49.096266 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:49.096875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:49.097155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:49.097429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:49.106904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:49.112721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:49.118353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1000/1991] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:49.196702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:49.198145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:49.258043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:49.266865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08788d9d-2edd-4e1d-bee0-509159a324b3 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:49.271118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:49.271453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e354-5103-41d7-ba35-73651f2c583d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:49.271849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:49.272224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:49.272581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:49.274354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:49.276175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:49.276813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:49.277213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:49.277488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] PATH_INFO: `/v3/projects/52daecc0758a40dfaa0ae239a6481ca9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:49.278575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52daecc0758a40dfaa0ae239a6481ca9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:49.280643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:49.334569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:49.336016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:49.345264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1001/1992] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:49.409243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52daecc0758a40dfaa0ae239a6481ca9', 'request_id': 'req-7fd22fe0-a774-4b76-bc37-2004a9bdd857'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.498913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52daecc0758a40dfaa0ae239a6481ca9', 'request_id': 'req-7fd22fe0-a774-4b76-bc37-2004a9bdd857'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.506199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52daecc0758a40dfaa0ae239a6481ca9', 'request_id': 'req-7fd22fe0-a774-4b76-bc37-2004a9bdd857'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.519526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] The token cache is being invalidate because project 52daecc0758a40dfaa0ae239a6481ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:49.519915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.520589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.523498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 992/1993] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:49 2022] DELETE /identity/v3/projects/52daecc0758a40dfaa0ae239a6481ca9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:49.532271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:49.539793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d573cf7-ceca-42ae-be09-0c2f0ff53f0e service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:49.576111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:49.604083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:49.604665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:49.604944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:49.605218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] PATH_INFO: `/v3/users/3d845cbce19545c1996b14c71c86dae5` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:49.606373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d845cbce19545c1996b14c71c86dae5)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:49.608723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:49.889806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:49.891311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:49.894349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d845cbce19545c1996b14c71c86dae5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.901651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:49.907695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd22fe0-a774-4b76-bc37-2004a9bdd857 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:49.914770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:49.921899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:49.921899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:49.921899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] PATH_INFO: `/v3/users/e2072a3d177043baaaa7a90eb8be7350` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:49.921899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2072a3d177043baaaa7a90eb8be7350)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:49.921899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:49.955819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d845cbce19545c1996b14c71c86dae5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.964629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d845cbce19545c1996b14c71c86dae5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.965332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d845cbce19545c1996b14c71c86dae5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.966023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-2948e5aa-a98c-432a-8bfd-d6c7ac60e7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d845cbce19545c1996b14c71c86dae5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:49.985001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1002/1994] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:49 2022] DELETE /identity/v3/users/3d845cbce19545c1996b14c71c86dae5 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:50.119012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2072a3d177043baaaa7a90eb8be7350'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.119868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2072a3d177043baaaa7a90eb8be7350'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.120667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2072a3d177043baaaa7a90eb8be7350'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.213724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2072a3d177043baaaa7a90eb8be7350'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.221240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2072a3d177043baaaa7a90eb8be7350'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.245039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 993/1995] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:49 2022] DELETE /identity/v3/users/e2072a3d177043baaaa7a90eb8be7350 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:50.246622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.247810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.247810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.268397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:50.269604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:50.335951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1003/1996] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:50.346116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:50.347525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:50.398223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:50.402396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:50.414995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe6a89-1937-4806-a566-38f2c388709c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:50.418663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375423d7-915b-4e9a-b4c4-2b4ec7bd2e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:50.419286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375423d7-915b-4e9a-b4c4-2b4ec7bd2e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.419578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375423d7-915b-4e9a-b4c4-2b4ec7bd2e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.419876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375423d7-915b-4e9a-b4c4-2b4ec7bd2e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.423593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd147f54-45a0-425e-b24f-290b631cc788 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:50.429089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:50.429741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.430005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.430279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] PATH_INFO: `/v3/projects/9490ebca87b34fd0b268a08722388432` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.431404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9490ebca87b34fd0b268a08722388432)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:50.432322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375423d7-915b-4e9a-b4c4-2b4ec7bd2e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:50.433377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:50.434153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375423d7-915b-4e9a-b4c4-2b4ec7bd2e57 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:50.444135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 994/1997] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:50.549597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:50.553392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9490ebca87b34fd0b268a08722388432', 'request_id': 'req-1e6ad691-ce20-4f21-9391-a56070823502'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.642604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9490ebca87b34fd0b268a08722388432', 'request_id': 'req-1e6ad691-ce20-4f21-9391-a56070823502'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.649858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9490ebca87b34fd0b268a08722388432', 'request_id': 'req-1e6ad691-ce20-4f21-9391-a56070823502'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.661101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] The token cache is being invalidate because project 9490ebca87b34fd0b268a08722388432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:50.661420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.669245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-1e6ad691-ce20-4f21-9391-a56070823502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:50.671697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1004/1998] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:50 2022] DELETE /identity/v3/projects/9490ebca87b34fd0b268a08722388432 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:50.676082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.676395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.676671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.696949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:50.698174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:50.760746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 995/1999] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:50.768752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.769127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.769402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.791568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:50.792790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:50.800864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1005/2000] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:50.805746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:50.815262 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78a160-379b-46e8-a386-faf20f1afbab None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:50.820550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:50.821145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.821419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.821687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.822324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:50.822623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:50.824255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:50.835818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 996/2001] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:50.840400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:50.849844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f497-747c-45cc-a1b9-2af6b3e01d95 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:50.854639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:50.858528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.858528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.858528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.858528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:50.858528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:50.905238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:50.907041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:50.917483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:50.921823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6a790-943d-4def-a9b8-d4b63ae9284c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:50.927527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:50.929850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:50.929850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:50.929850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] PATH_INFO: `/v3/projects/9b615ab48a5b4e31824a7e508790fd48` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:50.930322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b615ab48a5b4e31824a7e508790fd48)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:50.932421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:50.940244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1006/2002] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:50 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:50.941086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.014279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b9b5-b751-45e8-9b88-8a4fd7ef6257 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.019328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.019931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.020239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.020614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.021356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.023297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.048470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b615ab48a5b4e31824a7e508790fd48', 'request_id': 'req-c535305f-0a40-4c19-8949-33783911ef4f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.117091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1007/2003] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:50 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:51.121109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.131633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587b5f4-7807-4c0a-9a0d-cc45aeea6254 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.162431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b615ab48a5b4e31824a7e508790fd48', 'request_id': 'req-c535305f-0a40-4c19-8949-33783911ef4f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.170180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b615ab48a5b4e31824a7e508790fd48', 'request_id': 'req-c535305f-0a40-4c19-8949-33783911ef4f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.186932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] The token cache is being invalidate because project 9b615ab48a5b4e31824a7e508790fd48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:51.187226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.187929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.193439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 997/2004] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:50 2022] DELETE /identity/v3/projects/9b615ab48a5b4e31824a7e508790fd48 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:51.193439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.194186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:51.218096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.219251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.219553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.220102 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.222158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.223107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.241514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1008/2005] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:51.252436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.254632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:51.272208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c10c6-9df9-47a9-8c90-93d2ef0a5559 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.277201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.278086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.278433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.278756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] PATH_INFO: `/v3/users/ea39ec2168ef4a5596d032b6ac513cfc` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.279989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea39ec2168ef4a5596d032b6ac513cfc)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.282192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.294113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535305f-0a40-4c19-8949-33783911ef4f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.299084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.299845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.300264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.300607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] PATH_INFO: `/v3/users/62e34efd21e74468b53a267f28364dee` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.301873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e34efd21e74468b53a267f28364dee)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.304702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.356878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:51.428545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea39ec2168ef4a5596d032b6ac513cfc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.472733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e34efd21e74468b53a267f28364dee'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.473411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e34efd21e74468b53a267f28364dee'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.474037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e34efd21e74468b53a267f28364dee'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.482456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea39ec2168ef4a5596d032b6ac513cfc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.494147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea39ec2168ef4a5596d032b6ac513cfc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.494819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea39ec2168ef4a5596d032b6ac513cfc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.495439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea39ec2168ef4a5596d032b6ac513cfc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.512646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1009/2006] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:51 2022] DELETE /identity/v3/users/ea39ec2168ef4a5596d032b6ac513cfc => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:51.513432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.514751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:51.572091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e34efd21e74468b53a267f28364dee'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.579770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-45390e2a-0c69-40ea-9638-683b8cc86bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e34efd21e74468b53a267f28364dee'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:51.589540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657580-2985-4e72-9afd-7cd747b28112 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.596510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.597126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.597403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.597674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.598320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.600017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.601780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 998/2007] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:51 2022] DELETE /identity/v3/users/62e34efd21e74468b53a267f28364dee => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:51.607115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.607115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.607115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.616547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1010/2008] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:51.617477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.621190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:51.627665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:51.628958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:51.635783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8fc0e-f50d-4ae9-8645-3164b1db7c0d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.694896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.695643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.695990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.696354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] PATH_INFO: `/v3/projects/0ae5197798124832814db8d2591669e5/users/4bb0e281f2ff4530a6899364074132c1/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.701814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 999/2009] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:51.713057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae5197798124832814db8d2591669e5, user_id=4bb0e281f2ff4530a6899364074132c1, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.714039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.715593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:51.715713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.777956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcec9-3727-4183-8657-dac27efa4b5f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:51.781850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe870f5-cfce-49a7-8357-773d3091b1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:51.782538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe870f5-cfce-49a7-8357-773d3091b1db service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.782890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe870f5-cfce-49a7-8357-773d3091b1db service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.783259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe870f5-cfce-49a7-8357-773d3091b1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.793348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe870f5-cfce-49a7-8357-773d3091b1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:51.794821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe870f5-cfce-49a7-8357-773d3091b1db service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:51.804569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1000/2010] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:51.854686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1011/2011] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:51 2022] PUT /identity/v3/projects/0ae5197798124832814db8d2591669e5/users/4bb0e281f2ff4530a6899364074132c1/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:51.870640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:51.870928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:51.871198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:51.906888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] MFA Rules not processed for user `4bb0e281f2ff4530a6899364074132c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:51.908305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:51.982056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:51.985347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.008176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1001/2012] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:52.009700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.024128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:52.066325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c129c-4d27-4510-91ee-c1c94708649d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.073397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.074397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.074758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.074999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] PATH_INFO: `/v3/projects/6939a492744846cdbacb4e5c3b19c9ee` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.076350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6939a492744846cdbacb4e5c3b19c9ee)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.078306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.085191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afddb6-5f67-4fd0-a8a1-5c61a9b8d23e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.090146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.090780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.091225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.091491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.092123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.094105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.185380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6939a492744846cdbacb4e5c3b19c9ee', 'request_id': 'req-cc553a26-360c-43d5-8a22-e5c3d27810ae'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.222902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1002/2013] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:52.226814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.282283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6939a492744846cdbacb4e5c3b19c9ee', 'request_id': 'req-cc553a26-360c-43d5-8a22-e5c3d27810ae'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.298539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6939a492744846cdbacb4e5c3b19c9ee', 'request_id': 'req-cc553a26-360c-43d5-8a22-e5c3d27810ae'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.327378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7f9e6-1384-4979-8d73-64c70608f887 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.330015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] The token cache is being invalidate because project 6939a492744846cdbacb4e5c3b19c9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:52.330344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.331386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.333428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1012/2014] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:51 2022] DELETE /identity/v3/projects/6939a492744846cdbacb4e5c3b19c9ee => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:52.334460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.337835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de07b28-3fa8-4d30-a53b-9f6827ffaa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.338009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.338646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07b28-3fa8-4d30-a53b-9f6827ffaa89 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.338995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07b28-3fa8-4d30-a53b-9f6827ffaa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.339379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07b28-3fa8-4d30-a53b-9f6827ffaa89 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.340079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de07b28-3fa8-4d30-a53b-9f6827ffaa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.342465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de07b28-3fa8-4d30-a53b-9f6827ffaa89 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.396988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1003/2015] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:52.398785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.399076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.399353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.423477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc553a26-360c-43d5-8a22-e5c3d27810ae admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.424044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:52.425058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.429678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.430713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.431423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.431823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] PATH_INFO: `/v3/projects/dd3cb76a9a344c7bb8af35f93465c3ee` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.433121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd3cb76a9a344c7bb8af35f93465c3ee)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.435162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.435571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1004/2016] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:52.444273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.445788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.458236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc148a-f23d-4714-b58d-c582752350a1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.463476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ce6d-63b3-460f-8c9d-57f1b55eb5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.464254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ce6d-63b3-460f-8c9d-57f1b55eb5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.464605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ce6d-63b3-460f-8c9d-57f1b55eb5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.464950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ce6d-63b3-460f-8c9d-57f1b55eb5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.465672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ce6d-63b3-460f-8c9d-57f1b55eb5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.467312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ce6d-63b3-460f-8c9d-57f1b55eb5e2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.480205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1005/2017] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:52.480205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.480205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.480205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.502823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:52.503489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.516985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1006/2018] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:52.518699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.534760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075f604-de35-4a75-9381-32d37abcf955 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.540817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd3cb76a9a344c7bb8af35f93465c3ee', 'request_id': 'req-b47f396b-c55a-4efb-95c2-168590de3ff6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.606801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.607587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.608174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.608527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.609244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.610987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.619057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd3cb76a9a344c7bb8af35f93465c3ee', 'request_id': 'req-b47f396b-c55a-4efb-95c2-168590de3ff6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.626311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1007/2019] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:52.626311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.630131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd3cb76a9a344c7bb8af35f93465c3ee', 'request_id': 'req-b47f396b-c55a-4efb-95c2-168590de3ff6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.634242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3251ed3-5b32-4131-8f10-f76109646c6f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.640198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.640198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.640712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.641086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.642117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:52.642489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.644501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee166a-657a-409f-ba58-5723b9c4915f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.651439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] The token cache is being invalidate because project dd3cb76a9a344c7bb8af35f93465c3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:52.653743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.653743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:52.654758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1013/2020] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:52 2022] DELETE /identity/v3/projects/dd3cb76a9a344c7bb8af35f93465c3ee => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:52.655683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.658479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.660673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1008/2021] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:52.661928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.662735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.663252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.688477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f396b-c55a-4efb-95c2-168590de3ff6 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.689196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:52.689196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.689874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6309ba-797a-4d73-a162-28b9d505be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.690611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6309ba-797a-4d73-a162-28b9d505be7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.690959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6309ba-797a-4d73-a162-28b9d505be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.691286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6309ba-797a-4d73-a162-28b9d505be7a admin admin] PATH_INFO: `/v3/projects/c890eca7df3145fdae3c7827557dcf9d/users/6d659933382f4aeeafb1831483e3bd3a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.717657 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6309ba-797a-4d73-a162-28b9d505be7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c890eca7df3145fdae3c7827557dcf9d, user_id=6d659933382f4aeeafb1831483e3bd3a, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.717782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1009/2022] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:52.717782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.719029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.725732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6309ba-797a-4d73-a162-28b9d505be7a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.731362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:52.736188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07957ae0-e04b-4fdb-9a01-f689dd6f3883 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.742193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5237da2-a11d-42ab-b5ac-621d467f72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:52.742818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5237da2-a11d-42ab-b5ac-621d467f72e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.743107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5237da2-a11d-42ab-b5ac-621d467f72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.743405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5237da2-a11d-42ab-b5ac-621d467f72e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.752973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5237da2-a11d-42ab-b5ac-621d467f72e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:52.754923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5237da2-a11d-42ab-b5ac-621d467f72e3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:52.825435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1014/2023] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:52 2022] PUT /identity/v3/projects/c890eca7df3145fdae3c7827557dcf9d/users/6d659933382f4aeeafb1831483e3bd3a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:52.826905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.827195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.828449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.851271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:52.852673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.857713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1010/2024] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:52.858758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:52.859039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:52.859308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:52.894988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] MFA Rules not processed for user `6d659933382f4aeeafb1831483e3bd3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:52.896230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:52.917140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1015/2025] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:52.918242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:52.929003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951304-8792-4822-8fe2-143cde4d9911 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:52.935997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:52.978286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1011/2026] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:52.981559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.014184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:53.017808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.017808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.017808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.017808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.018129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.018913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.051080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97866a73-3396-469d-93cf-ba8a479752c0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.055826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.056462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.056744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.057014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.057655 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:53.057958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.059561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.070717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1012/2027] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:53.075267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.080771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93aca-c687-4247-b91d-ed62973645c5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.085902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.086457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.086741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.087010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.087617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.089489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.123336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1016/2028] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:52 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:53.124343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.133906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a575af-d8ee-4a56-92eb-498f5c8d12a1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.138976 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.139604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.139882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.140174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.140801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.143104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.177705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1013/2029] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:53 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:53.177705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.241740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312514-2b13-4d99-99e8-48b8bb99323e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.247928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.248612 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.248914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.249197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.250042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.252039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.258232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1017/2030] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:53 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:53.259530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.264787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1014/2031] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:53.265730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.276355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52430f11-e55e-4305-be6c-71829d73754a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.329057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a670894-c104-4174-89c0-d327aa651758 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.334584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.335524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.335524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.335731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.336394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.338335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.339291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.340389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.340389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.340600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.341080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.343379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.401916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:53.407590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1018/2032] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:53 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:53.408456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.417697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241697-49c3-44a7-9a59-19d7324208a3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.422404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.423007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.423276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.423543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.424193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.426133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.435685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1019/2033] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:53.436471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.446755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473392a-1974-46c6-a2cd-e61a44ac16d8 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.451701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.452325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.452599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.452915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.453498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.455088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.467084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1020/2034] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:53.467842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.477009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980d80c-9394-4187-94cf-8902c9b63fdd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.481628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.482235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.482508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.482782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] PATH_INFO: `/v3/projects/2ed99ce2509c4163b314baa4cf3e5ae5/users/cfd87fa14f924cb897959950c79739ef/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.487691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1015/2035] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:53 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:53.497849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.497965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed99ce2509c4163b314baa4cf3e5ae5, user_id=cfd87fa14f924cb897959950c79739ef, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.498219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06f131-3291-49f3-a287-56b97d684b1d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.500686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.503778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92c4aa-4bd8-4b38-92e4-ba338a663f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.504544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92c4aa-4bd8-4b38-92e4-ba338a663f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.504825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92c4aa-4bd8-4b38-92e4-ba338a663f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.505099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92c4aa-4bd8-4b38-92e4-ba338a663f39 admin admin] PATH_INFO: `/v3/projects/666a453926e0461b8638e173f8a32427/users/84411ca8977143e1bd1678935056bf1e/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.519283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:53.521366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92c4aa-4bd8-4b38-92e4-ba338a663f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666a453926e0461b8638e173f8a32427, user_id=84411ca8977143e1bd1678935056bf1e, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.524697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92c4aa-4bd8-4b38-92e4-ba338a663f39 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.585646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1021/2036] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:53 2022] PUT /identity/v3/projects/2ed99ce2509c4163b314baa4cf3e5ae5/users/cfd87fa14f924cb897959950c79739ef/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:53.586582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.603974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1016/2037] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:53 2022] PUT /identity/v3/projects/666a453926e0461b8638e173f8a32427/users/84411ca8977143e1bd1678935056bf1e/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:53.607152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.607152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.607152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.626024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:53.627253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:53.649029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5464d70-bbd1-442d-b824-10f0e2e3bdc7 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.688866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1017/2038] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:53.689884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.699834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29362dd-1cfc-4516-886d-886396b82e06 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.704573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.705118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.705400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.705673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.705987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.706338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.706681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.706961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.707232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.707868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.707971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.710148 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.719415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1018/2039] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:53.720533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.723602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1022/2040] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:53.723726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:53.726841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.737574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf6b54-d872-4401-ad9c-8a78c274e040 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.742954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.743703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.743983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.744344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] PATH_INFO: `/v3/projects/666a453926e0461b8638e173f8a32427/users/84411ca8977143e1bd1678935056bf1e/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.746635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666a453926e0461b8638e173f8a32427, user_id=84411ca8977143e1bd1678935056bf1e, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.749145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.761506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1023/2041] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:53 2022] PUT /identity/v3/projects/666a453926e0461b8638e173f8a32427/users/84411ca8977143e1bd1678935056bf1e/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:53.762361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.808853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fe17d-3fce-401f-b994-3105f7cc6512 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.835643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5896c69-063e-407a-b6ad-546b3f21dd25 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.841357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed685d-7557-47ae-bc82-577f9538bcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.842030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed685d-7557-47ae-bc82-577f9538bcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.842330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed685d-7557-47ae-bc82-577f9538bcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.842603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed685d-7557-47ae-bc82-577f9538bcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.843254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed685d-7557-47ae-bc82-577f9538bcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.845172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed685d-7557-47ae-bc82-577f9538bcd7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.866066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1024/2042] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:53.868062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.868062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.868062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.877670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.878929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.879418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.879765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.892505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.894091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.898875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] MFA Rules not processed for user `84411ca8977143e1bd1678935056bf1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:53.900095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:53.904285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1019/2043] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:53.905149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.914682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd2f55-1d09-462d-bbc6-c448ac4f35b8 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:53.920168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8ad99-7d91-4bd4-82c0-a6709a48da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:53.921569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8ad99-7d91-4bd4-82c0-a6709a48da25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:53.924491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:53.924491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8ad99-7d91-4bd4-82c0-a6709a48da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:53.924715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8ad99-7d91-4bd4-82c0-a6709a48da25 admin admin] PATH_INFO: `/v3/projects/dcbe60d2c893418aaba65a6fc3ed3776/users/acedbdbf7c6743788262af5168274991/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:53.940865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8ad99-7d91-4bd4-82c0-a6709a48da25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbe60d2c893418aaba65a6fc3ed3776, user_id=acedbdbf7c6743788262af5168274991, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:53.944173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8ad99-7d91-4bd4-82c0-a6709a48da25 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:53.972345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1025/2044] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:53.983990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:53.985426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.031380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1020/2045] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:53 2022] PUT /identity/v3/projects/dcbe60d2c893418aaba65a6fc3ed3776/users/acedbdbf7c6743788262af5168274991/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:54.042131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.042419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.042698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.046588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7988b5-6650-4321-a3db-fc3e4ab77564 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.076620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] MFA Rules not processed for user `acedbdbf7c6743788262af5168274991`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:54.077858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.105472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.106251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.106440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.107226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.147617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1021/2046] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:54.151782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.169592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.171040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.180730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1026/2047] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:54.208272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:54.216363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e5da4-b697-4685-950c-4a3d69455901 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.220546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.221168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.221450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.221725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.222430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.224313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.312593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.322153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc496b-7e60-4b5f-83cf-a204c0d6c43f service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.326051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.330513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.330513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.330513 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.330807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1022/2048] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:54 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:54.331239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.333364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.392474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.394737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.404406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb204c-32a5-40a5-9619-6fdcba2a4aec admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.406149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1027/2049] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:54.407170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.408529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.413530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50434501-f091-4800-8342-51f829811e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.415999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50434501-f091-4800-8342-51f829811e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.415999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50434501-f091-4800-8342-51f829811e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.415999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50434501-f091-4800-8342-51f829811e21 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.416794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50434501-f091-4800-8342-51f829811e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.419288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50434501-f091-4800-8342-51f829811e21 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.431635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafe4f3-eed7-4676-b803-a0060786912f service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.445358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.446081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.448819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.448819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] PATH_INFO: `/v3/users/fd4c0f6a4eda4a4a9d3bea9b5ac4610d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.448819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4c0f6a4eda4a4a9d3bea9b5ac4610d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.451199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.476630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:54.546270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1023/2050] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:54 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:54.549992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50434501-f091-4800-8342-51f829811e21 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.561369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50434501-f091-4800-8342-51f829811e21 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.566784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.567603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.568000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.568947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.573446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.587892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.609539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1024/2051] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:54.613468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.623821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d8c12-c563-4d73-a7d7-3bd0b15c8d07 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.628397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.628984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.629256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.629522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.630214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.632436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.646301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1025/2052] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:54.647391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.657328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0ebda-b6e8-43f2-9a25-0e34a681ee51 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.662376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864db7ca-f3f1-4ccc-892c-c2326ade6da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.663035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864db7ca-f3f1-4ccc-892c-c2326ade6da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.663366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864db7ca-f3f1-4ccc-892c-c2326ade6da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.663651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864db7ca-f3f1-4ccc-892c-c2326ade6da9 admin admin] PATH_INFO: `/v3/projects/b6fbaaf583bc44f88dc5d08ed1b65894/users/78fcb8bde16d4d0d92cc1e47b7eaa8f8/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.678474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864db7ca-f3f1-4ccc-892c-c2326ade6da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fbaaf583bc44f88dc5d08ed1b65894, user_id=78fcb8bde16d4d0d92cc1e47b7eaa8f8, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.681324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864db7ca-f3f1-4ccc-892c-c2326ade6da9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.704008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4c0f6a4eda4a4a9d3bea9b5ac4610d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:54.741268 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1026/2053] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:54 2022] PUT /identity/v3/projects/b6fbaaf583bc44f88dc5d08ed1b65894/users/78fcb8bde16d4d0d92cc1e47b7eaa8f8/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:54.751263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.751694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.752193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.762614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4c0f6a4eda4a4a9d3bea9b5ac4610d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:54.773390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4c0f6a4eda4a4a9d3bea9b5ac4610d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:54.774570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4c0f6a4eda4a4a9d3bea9b5ac4610d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:54.774865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-07fc682d-67da-4967-b55c-be92bde4c467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4c0f6a4eda4a4a9d3bea9b5ac4610d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:54.783805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] MFA Rules not processed for user `78fcb8bde16d4d0d92cc1e47b7eaa8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:54.785113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.797225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1028/2054] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:54 2022] DELETE /identity/v3/users/fd4c0f6a4eda4a4a9d3bea9b5ac4610d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:54.802172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.802464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.803319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.827264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:54.829089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.868480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1027/2055] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:54.872994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.874368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.899532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1029/2056] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:54.909299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:54.911179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:54.962297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9b7cb-ce72-40c7-8418-ca8591754307 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.967753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.968408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.968692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.968968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:54.969788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:54.975073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:54.987997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede920d6-149e-460d-b190-b4fa5a1eec1e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:54.993176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:54.993731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:54.994006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:54.994270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.003654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.005137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.011613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:55.017076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1030/2057] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:55.123924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1028/2058] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:54 2022] POST /identity/v3/projects => generated 383 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:46:55.128137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.138775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd9fda-ff85-4b44-b904-97245ed738d3 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.203420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.204295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.204646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.205012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.205832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.207932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.294265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:55.300169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1031/2059] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:55 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:55.306589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.327690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc6142-b17f-4d85-8732-042a6d8d03e8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.331845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d85a1d-bdaf-4e8a-8be7-68ef9ff70835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.332604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d85a1d-bdaf-4e8a-8be7-68ef9ff70835 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.332961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d85a1d-bdaf-4e8a-8be7-68ef9ff70835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.333303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d85a1d-bdaf-4e8a-8be7-68ef9ff70835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.336186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d85a1d-bdaf-4e8a-8be7-68ef9ff70835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.336455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d85a1d-bdaf-4e8a-8be7-68ef9ff70835 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.363253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1029/2060] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:55.370887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.395599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81264011-260d-45e5-9357-5b6219c1a05c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.403967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.404838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.405237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.405580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] PATH_INFO: `/v3/projects/1c80f7f852aa4658b06a8469a350fb31/users/5372d4a4b0af4ccc92743b8d3178c2c5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.423328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c80f7f852aa4658b06a8469a350fb31, user_id=5372d4a4b0af4ccc92743b8d3178c2c5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.426367 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.430864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:55.467548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225731-8960-457b-908b-5268ee296d89 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.467947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225731-8960-457b-908b-5268ee296d89 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.468342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225731-8960-457b-908b-5268ee296d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.503014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e2225731-8960-457b-908b-5268ee296d89 None None] MFA Rules not processed for user `cfd87fa14f924cb897959950c79739ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:55.505407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e2225731-8960-457b-908b-5268ee296d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:55.528866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1032/2061] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:55 2022] PUT /identity/v3/projects/1c80f7f852aa4658b06a8469a350fb31/users/5372d4a4b0af4ccc92743b8d3178c2c5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:55.533126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.592866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1030/2062] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:55.597623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225731-8960-457b-908b-5268ee296d89 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.600221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e2225731-8960-457b-908b-5268ee296d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:55.621031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225731-8960-457b-908b-5268ee296d89 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.661137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2190e25-f9d7-43b3-b043-ade813d6772c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.667796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.668446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.668731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.669206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.669870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.671497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.689465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1033/2063] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:55.690254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.692547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:55.712175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.712834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.713112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.713403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] PATH_INFO: `/v3/projects/2730501ba82d4c54be8fcccc2d597c7d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.715253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2730501ba82d4c54be8fcccc2d597c7d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.717300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.756769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae725d-27b9-4315-b9a2-0b25a9f36eee admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.760561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.761167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.761445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.761717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.823173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.824880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.835084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1034/2064] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:55.836435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.847651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2730501ba82d4c54be8fcccc2d597c7d', 'request_id': 'req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:55.850840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d00f9-0cc4-4bfe-b36e-7a067270faf1 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:55.926601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38422-7774-4ad0-b170-b04c4dea1673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:55.927348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38422-7774-4ad0-b170-b04c4dea1673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.927567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38422-7774-4ad0-b170-b04c4dea1673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.928638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38422-7774-4ad0-b170-b04c4dea1673 admin admin] PATH_INFO: `/v3/projects/1c80f7f852aa4658b06a8469a350fb31/users/5372d4a4b0af4ccc92743b8d3178c2c5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.931250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38422-7774-4ad0-b170-b04c4dea1673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c80f7f852aa4658b06a8469a350fb31, user_id=5372d4a4b0af4ccc92743b8d3178c2c5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:55.934701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38422-7774-4ad0-b170-b04c4dea1673 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:55.951159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2730501ba82d4c54be8fcccc2d597c7d', 'request_id': 'req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:55.951692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1035/2065] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:55 2022] PUT /identity/v3/projects/1c80f7f852aa4658b06a8469a350fb31/users/5372d4a4b0af4ccc92743b8d3178c2c5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:55.957827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2730501ba82d4c54be8fcccc2d597c7d', 'request_id': 'req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:55.965076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:55.966184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:55.966811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:55.970522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] The token cache is being invalidate because project 2730501ba82d4c54be8fcccc2d597c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:55.970900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:55.972077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:55.973796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1031/2066] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:55 2022] DELETE /identity/v3/projects/2730501ba82d4c54be8fcccc2d597c7d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:55.985364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:55.986764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:56.000876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1cd4-3c95-48d7-8398-62d4ac38ec2d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:56.006812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] MFA Rules not processed for user `5372d4a4b0af4ccc92743b8d3178c2c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:56.014652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:56.031843 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:56.066843 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:56.067433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:56.067712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:56.067989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] PATH_INFO: `/v3/users/65584c1496624848b20f2c64ef88f3ed` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:56.069719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65584c1496624848b20f2c64ef88f3ed)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:56.071645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:56.094632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1036/2067] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:56.106169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:56.107692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:56.164921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c6e43-9000-451b-b20e-433542fe2e7b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:56.168552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:56.169157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:56.169436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:56.169707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:56.171437 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:56.229583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:56.231046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:56.239941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1037/2068] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:46:56.436113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65584c1496624848b20f2c64ef88f3ed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:56.437035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65584c1496624848b20f2c64ef88f3ed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:56.437556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65584c1496624848b20f2c64ef88f3ed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:56.506982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65584c1496624848b20f2c64ef88f3ed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:56.522537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6ed842ac-c850-4049-ab1c-04b554dfe5e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65584c1496624848b20f2c64ef88f3ed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:56.540811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1032/2069] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:55 2022] DELETE /identity/v3/users/65584c1496624848b20f2c64ef88f3ed => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:56.945074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:57.043291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.060168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207145c-d816-434e-bd8e-cbd7d8386bb1 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.152659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.153308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.153591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.153865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] PATH_INFO: `/v3/projects/d1f990daabf24426998af05b1a17a8c1` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.154999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f990daabf24426998af05b1a17a8c1)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.156935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:57.283172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f990daabf24426998af05b1a17a8c1', 'request_id': 'req-f840ca71-fdcb-4378-85bb-e2f8e3873a65'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:57.390230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f990daabf24426998af05b1a17a8c1', 'request_id': 'req-f840ca71-fdcb-4378-85bb-e2f8e3873a65'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:57.398412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f990daabf24426998af05b1a17a8c1', 'request_id': 'req-f840ca71-fdcb-4378-85bb-e2f8e3873a65'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:57.410332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] The token cache is being invalidate because project d1f990daabf24426998af05b1a17a8c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:46:57.410624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:57.411791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f840ca71-fdcb-4378-85bb-e2f8e3873a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:57.413631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1038/2070] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:46:57 2022] DELETE /identity/v3/projects/d1f990daabf24426998af05b1a17a8c1 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:57.418688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.418981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.419259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.439583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:57.441911 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:57.511071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1033/2071] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:57.514458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.514802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.515584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.537521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:57.538784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:57.547188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1039/2072] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:57.551532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.562099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7b91c-e858-44e2-ad18-9f20d97bd5c4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.566747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.567352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.567628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.567897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.569183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:46:57.573290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.573290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:57.583236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1034/2073] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:46:57.587897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.597542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b0c0-acbe-4fa0-9ee3-4b85b26dff1b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.602776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e533d-6272-4197-b717-e3aa813eeee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.603426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e533d-6272-4197-b717-e3aa813eeee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.603720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e533d-6272-4197-b717-e3aa813eeee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.603990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e533d-6272-4197-b717-e3aa813eeee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.604667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e533d-6272-4197-b717-e3aa813eeee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.606722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e533d-6272-4197-b717-e3aa813eeee5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:57.687931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1040/2074] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:57 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:57.689331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.700324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81d27e-90eb-4ecf-a00d-4ddf58c292f7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.769999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.770672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.770954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.771229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.771960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.773967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:57.776178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.776609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.776890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.796575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] MFA Rules not processed for user `cfd87fa14f924cb897959950c79739ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:57.797825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:57.854153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1041/2075] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:46:57.871716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1035/2076] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:57 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:57.873682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.886680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae74e25-73e0-43bc-b2e1-c43e3460055a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.895681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.895681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.895681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.895681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.895961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:57.896047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.896047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:57.907064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1042/2077] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:46:57.911145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.928798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457960a2-d1a5-48ec-8526-f2e9281aacd2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.928798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b527e0-ad53-423a-a561-12c3600ef4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.928956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b527e0-ad53-423a-a561-12c3600ef4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.929125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b527e0-ad53-423a-a561-12c3600ef4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.929392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b527e0-ad53-423a-a561-12c3600ef4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.930046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b527e0-ad53-423a-a561-12c3600ef4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.931642 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b527e0-ad53-423a-a561-12c3600ef4a3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:57.942765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1036/2078] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:57.947197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:57.957334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eb2b8-9820-4c93-b8ea-032d376d2527 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:57.962042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:57.962677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:57.962952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:57.963223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] PATH_INFO: `/v3/projects/da9312de4a924251896cf79d6e2f5993/users/5da4c71393a34e2c82a9e6b2653c6e04/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:57.987084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9312de4a924251896cf79d6e2f5993, user_id=5da4c71393a34e2c82a9e6b2653c6e04, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:57.989853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.058553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1043/2079] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:57 2022] PUT /identity/v3/projects/da9312de4a924251896cf79d6e2f5993/users/5da4c71393a34e2c82a9e6b2653c6e04/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:58.065803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:58.077574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.077574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.077574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.128136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] MFA Rules not processed for user `5da4c71393a34e2c82a9e6b2653c6e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:58.129536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:58.223645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1037/2080] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:58.228104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.308653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e405562-6032-46ad-96f4-1bd2f913ad83 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.313880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.318058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.319315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.319934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.320366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.322501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.419275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1044/2081] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:46:58 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:58.420651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.428264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.430423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:58.433631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a0121-5b97-4d7e-b427-d804eca4232a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.494039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b402e48-74fc-4caf-a8b8-4f21156836f5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.495896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.496550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.496830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.497101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.497733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.498063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.498696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.499048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.499354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.499653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.501141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:58.545211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:58.565074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.567066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.577634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1045/2082] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:58.609245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1038/2083] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:46:58 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:46:58.612945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.624429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01128244-513f-4357-b61a-aaa2ce05413e service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.629993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.631421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.631853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.632256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.633048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.638471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.652799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1046/2084] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:58.657194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.669907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d5773-a716-4e0f-9830-82b3141c31f3 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.691647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.693924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.694602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.695498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] PATH_INFO: `/v3/projects/34732cabf0804bb48bf04fcb84283641/users/151f0052883941fc89686a85f4dbd849/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.720394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34732cabf0804bb48bf04fcb84283641, user_id=151f0052883941fc89686a85f4dbd849, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.724579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.830839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1039/2085] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:58 2022] PUT /identity/v3/projects/34732cabf0804bb48bf04fcb84283641/users/151f0052883941fc89686a85f4dbd849/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:58.835038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.848402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1ecff-a358-49c0-b315-fc9d476979d5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.908849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb1216-af10-4034-80b6-cbdd280506b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.909422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1216-af10-4034-80b6-cbdd280506b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.909705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1216-af10-4034-80b6-cbdd280506b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.910005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1216-af10-4034-80b6-cbdd280506b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.910698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb1216-af10-4034-80b6-cbdd280506b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.912911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb1216-af10-4034-80b6-cbdd280506b4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.923124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1047/2086] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:46:58.928370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:58.939723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac4dc2-2f0e-4b0f-97ee-5f79313f2be0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:58.944759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:58.945504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.945856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.946198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] PATH_INFO: `/v3/projects/34732cabf0804bb48bf04fcb84283641/users/151f0052883941fc89686a85f4dbd849/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:58.948355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34732cabf0804bb48bf04fcb84283641, user_id=151f0052883941fc89686a85f4dbd849, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:58.951090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:58.961959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1040/2087] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:46:58 2022] PUT /identity/v3/projects/34732cabf0804bb48bf04fcb84283641/users/151f0052883941fc89686a85f4dbd849/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:46:58.973787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:58.974202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:58.974290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:59.002707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] MFA Rules not processed for user `151f0052883941fc89686a85f4dbd849`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:59.003910 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:59.075961 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1048/2088] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:46:59.087488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:59.089249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:59.151710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6539275-79dc-442b-941c-6b03e352ed1f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:59.155501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb8a61-c224-4122-84af-7a8aec24ee6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:59.156112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb8a61-c224-4122-84af-7a8aec24ee6b service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:59.156419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb8a61-c224-4122-84af-7a8aec24ee6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:59.156693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb8a61-c224-4122-84af-7a8aec24ee6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:59.166682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb8a61-c224-4122-84af-7a8aec24ee6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:59.168164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb8a61-c224-4122-84af-7a8aec24ee6b service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:59.177792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1041/2089] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:46:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:46:59.218818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:59.282150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:46:59.484388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:46:59.485871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:46:59.553404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ff41-b16a-47ca-b303-e3d5ede37da1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:46:59.558097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:46:59.558723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:59.559003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:59.559277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] PATH_INFO: `/v3/users/4bb0e281f2ff4530a6899364074132c1` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:59.560515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bb0e281f2ff4530a6899364074132c1)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:46:59.565978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:46:59.837586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bb0e281f2ff4530a6899364074132c1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:59.915885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bb0e281f2ff4530a6899364074132c1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:59.925442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bb0e281f2ff4530a6899364074132c1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:59.926123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bb0e281f2ff4530a6899364074132c1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:59.926735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bb0e281f2ff4530a6899364074132c1'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:46:59.947928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1049/2090] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:46:59 2022] DELETE /identity/v3/users/4bb0e281f2ff4530a6899364074132c1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:46:59.954022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:46:59.954521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:46:59.954613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:46:59.993249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:46:59.994566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.084866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1042/2091] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:00.095084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:00.096953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.169312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0b40-8e47-4e89-9d4a-933d57c34a09 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:00.176843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:00.177697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:00.178083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:00.178364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:00.187944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:00.189480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:00.199778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1050/2092] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:00.212134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:00.213562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.250287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:00.288008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:00.289611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.291208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464e193-bbc9-40f6-b053-0b0935e9fdf8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:00.297057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:00.298255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:00.298595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:00.298871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:00.302743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.303542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639bfb0-b67e-4af0-9c06-c2c069dcfd08 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:00.309340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:00.309972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:00.310244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:00.310685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] PATH_INFO: `/v3/users/5da4c71393a34e2c82a9e6b2653c6e04` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:00.312228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da4c71393a34e2c82a9e6b2653c6e04)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:00.314502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:00.372981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:00.374420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708a0bf-e691-4bfa-9859-c73f43bf0b90 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:00.384439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1043/2093] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:00.505093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:00.539360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da4c71393a34e2c82a9e6b2653c6e04'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.590489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da4c71393a34e2c82a9e6b2653c6e04'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.598464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da4c71393a34e2c82a9e6b2653c6e04'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.599166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da4c71393a34e2c82a9e6b2653c6e04'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.599828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da4c71393a34e2c82a9e6b2653c6e04'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.617907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1051/2094] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:00 2022] DELETE /identity/v3/users/5da4c71393a34e2c82a9e6b2653c6e04 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:00.622290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:00.622638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:00.622849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:00.643216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:00.644447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.706334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:00.708336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.712689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1044/2095] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:00.727806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad522d2a-74a6-482e-a671-9952e2e1f258 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:00.732310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:00.732493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:00.733252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:00.733705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:00.733990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] PATH_INFO: `/v3/projects/0ae5197798124832814db8d2591669e5` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:00.734776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:00.735344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae5197798124832814db8d2591669e5)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:00.737621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:00.796558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190f5f6-5f77-4289-b65f-9eb6f29ea579 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:00.805249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:00.805249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:00.805249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:00.805249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:00.812167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:00.813668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:00.823252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1045/2096] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:00.867221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae5197798124832814db8d2591669e5', 'request_id': 'req-f694b5c6-a77e-4792-a43f-29d7f9a580b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.967746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae5197798124832814db8d2591669e5', 'request_id': 'req-f694b5c6-a77e-4792-a43f-29d7f9a580b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.974937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae5197798124832814db8d2591669e5', 'request_id': 'req-f694b5c6-a77e-4792-a43f-29d7f9a580b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.986172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] The token cache is being invalidate because project 0ae5197798124832814db8d2591669e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:00.986458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.987144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:00.988663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1052/2097] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:00 2022] DELETE /identity/v3/projects/0ae5197798124832814db8d2591669e5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:00.992803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:01.003564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84ef2d-63b0-42a3-bce4-f313fd0571a5 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:01.016164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:01.090170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:01.090170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:01.090170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:01.090170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] PATH_INFO: `/v3/users/6d659933382f4aeeafb1831483e3bd3a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:01.090928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d659933382f4aeeafb1831483e3bd3a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:01.091428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:01.334695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d659933382f4aeeafb1831483e3bd3a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.335423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d659933382f4aeeafb1831483e3bd3a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.336073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d659933382f4aeeafb1831483e3bd3a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.379491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:01.396231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d659933382f4aeeafb1831483e3bd3a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.404727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d659933382f4aeeafb1831483e3bd3a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.428179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1046/2098] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:00 2022] DELETE /identity/v3/users/6d659933382f4aeeafb1831483e3bd3a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:01.481257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:01.483681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:01.499107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b5c6-a77e-4792-a43f-29d7f9a580b3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:01.563689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:01.564407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:01.564661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:01.564932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] PATH_INFO: `/v3/projects/da9312de4a924251896cf79d6e2f5993` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:01.566069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9312de4a924251896cf79d6e2f5993)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:01.568020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:01.723003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9312de4a924251896cf79d6e2f5993', 'request_id': 'req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.820705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9312de4a924251896cf79d6e2f5993', 'request_id': 'req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.840326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9312de4a924251896cf79d6e2f5993', 'request_id': 'req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.852970 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] The token cache is being invalidate because project da9312de4a924251896cf79d6e2f5993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:01.853535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.854357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:01.856018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1053/2099] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:01 2022] DELETE /identity/v3/projects/da9312de4a924251896cf79d6e2f5993 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:01.860252 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:01.862154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:01.879412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d6a5-796f-459b-8f2f-1fdd8731d4c3 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:01.914667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:01.919133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:02.004044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:02.006742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:02.007220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:02.007879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] PATH_INFO: `/v3/users/151f0052883941fc89686a85f4dbd849` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:02.009274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=151f0052883941fc89686a85f4dbd849)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:02.011538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:02.033969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed81c3-7ea0-4fb3-915a-3d4e77a1379c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:02.039381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:02.040236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:02.040514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:02.044060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] PATH_INFO: `/v3/projects/c890eca7df3145fdae3c7827557dcf9d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:02.044060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c890eca7df3145fdae3c7827557dcf9d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:02.044060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:02.232697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:02.253238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c890eca7df3145fdae3c7827557dcf9d', 'request_id': 'req-282759c4-fd2e-4ddc-865f-53140584a83c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.262110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151f0052883941fc89686a85f4dbd849'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.263177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151f0052883941fc89686a85f4dbd849'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.263868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151f0052883941fc89686a85f4dbd849'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.357535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c890eca7df3145fdae3c7827557dcf9d', 'request_id': 'req-282759c4-fd2e-4ddc-865f-53140584a83c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.366267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c890eca7df3145fdae3c7827557dcf9d', 'request_id': 'req-282759c4-fd2e-4ddc-865f-53140584a83c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.379632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] The token cache is being invalidate because project c890eca7df3145fdae3c7827557dcf9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:02.380189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.381577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.384930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1054/2100] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:01 2022] DELETE /identity/v3/projects/c890eca7df3145fdae3c7827557dcf9d => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:02.387619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:02.389075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:02.454781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282759c4-fd2e-4ddc-865f-53140584a83c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:02.459545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151f0052883941fc89686a85f4dbd849'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.461865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:02.463521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:02.463826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:02.464103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] PATH_INFO: `/v3/users/84411ca8977143e1bd1678935056bf1e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:02.465311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84411ca8977143e1bd1678935056bf1e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:02.467476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:02.467797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151f0052883941fc89686a85f4dbd849'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.499912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1047/2101] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:01 2022] DELETE /identity/v3/users/151f0052883941fc89686a85f4dbd849 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:02.689488 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84411ca8977143e1bd1678935056bf1e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.801590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84411ca8977143e1bd1678935056bf1e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.813731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84411ca8977143e1bd1678935056bf1e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.814536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84411ca8977143e1bd1678935056bf1e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.815561 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fc1aa2f-f6a3-4477-a959-650cd222efe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84411ca8977143e1bd1678935056bf1e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:02.837997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1055/2102] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:02 2022] DELETE /identity/v3/users/84411ca8977143e1bd1678935056bf1e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:02.956803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:02.959555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:03.066392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820982-802f-4bb3-bfc1-43436d3974bf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:03.131855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:03.133427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.133946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.134430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] PATH_INFO: `/v3/projects/34732cabf0804bb48bf04fcb84283641` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.135827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34732cabf0804bb48bf04fcb84283641)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:03.138812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:03.248241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:03.266375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34732cabf0804bb48bf04fcb84283641', 'request_id': 'req-790f3886-d2bd-4fdf-9640-6ea6766d47d8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.310358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34732cabf0804bb48bf04fcb84283641', 'request_id': 'req-790f3886-d2bd-4fdf-9640-6ea6766d47d8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.317452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34732cabf0804bb48bf04fcb84283641', 'request_id': 'req-790f3886-d2bd-4fdf-9640-6ea6766d47d8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.332168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] The token cache is being invalidate because project 34732cabf0804bb48bf04fcb84283641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:03.332478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.333171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-790f3886-d2bd-4fdf-9640-6ea6766d47d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.334811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1048/2103] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:02 2022] DELETE /identity/v3/projects/34732cabf0804bb48bf04fcb84283641 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:03.339819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.340112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.340418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.368524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:03.370189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:03.450692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1056/2104] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:03.458909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.459200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.459530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.481676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:03.482944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:03.509901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:03.516959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1049/2105] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:03.522577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d085-b1f7-48ee-b274-6f972e2d1b6f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:03.529547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:03.530055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.530442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.531120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.532327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:03.532846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:03.535273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:03.555895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1057/2106] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:47:03.556981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:03.568167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95135e9c-f269-433d-a0aa-a6176e758bb1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:03.582405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:03.583171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.583545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.584490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.585386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:03.588562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:03.650215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:03.653540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:03.669951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af254f-ff27-4a45-90f1-14e53ae36afd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:03.674589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:03.675239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.675515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.675788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] PATH_INFO: `/v3/projects/666a453926e0461b8638e173f8a32427` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.676921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=666a453926e0461b8638e173f8a32427)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:03.679184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:03.705933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1050/2107] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:03 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:03.707427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:03.728415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:03.789436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4b055-5eeb-46ee-a8f2-379f9de61e33 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:03.794346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:03.795074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.795448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.795822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.796623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:03.798820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:03.831484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666a453926e0461b8638e173f8a32427', 'request_id': 'req-50e49891-19be-4609-a27e-7b6a20f9fd8e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.904210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1051/2108] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:03 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:03.906673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:03.917182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad58b87-98ca-423f-9fe8-79c94751fecb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:03.920871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666a453926e0461b8638e173f8a32427', 'request_id': 'req-50e49891-19be-4609-a27e-7b6a20f9fd8e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.928730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666a453926e0461b8638e173f8a32427', 'request_id': 'req-50e49891-19be-4609-a27e-7b6a20f9fd8e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.941432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] The token cache is being invalidate because project 666a453926e0461b8638e173f8a32427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:03.941809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.942862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-50e49891-19be-4609-a27e-7b6a20f9fd8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:03.945721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1058/2109] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:03 2022] DELETE /identity/v3/projects/666a453926e0461b8638e173f8a32427 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:03.952363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecd68a-a262-4805-9506-45dc1d112373 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.952516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecd68a-a262-4805-9506-45dc1d112373 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.952741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecd68a-a262-4805-9506-45dc1d112373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.988019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5fecd68a-a262-4805-9506-45dc1d112373 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:03.991103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5fecd68a-a262-4805-9506-45dc1d112373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:03.995965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:03.996643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:03.996928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:03.997206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:03.998628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.000420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.009578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1059/2110] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:04.011094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.011459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.011867 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.012753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1052/2111] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:04.016872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.019003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:04.031736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537618fb-ce5d-4137-a6a9-1cbe7f29811f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.037404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.038063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.038348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.038625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.039147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:04.039311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.040638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:04.041444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.051933 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1053/2112] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:04.054101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1060/2113] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:04.054960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.056709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.064338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ec637-34ca-430e-9913-4994364c82b7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.070702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8834ef-03b6-4aca-ad50-af4daafba722 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.071181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.071406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.071925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.072243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.072902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:04.073202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.075552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.079080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f666d6f-b7f4-45ea-b015-e373c74c4454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.079579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f666d6f-b7f4-45ea-b015-e373c74c4454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.079846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f666d6f-b7f4-45ea-b015-e373c74c4454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.080245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f666d6f-b7f4-45ea-b015-e373c74c4454 admin admin] PATH_INFO: `/v3/projects/eca8c248eefe4c78b78a3eaf4899cb0c/users/ea60e85bf5304df69c99ed1ae98c0ef3/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.095684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1061/2114] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:47:04.096651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.108129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f666d6f-b7f4-45ea-b015-e373c74c4454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca8c248eefe4c78b78a3eaf4899cb0c, user_id=ea60e85bf5304df69c99ed1ae98c0ef3, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.111453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f666d6f-b7f4-45ea-b015-e373c74c4454 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.119066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af552ecc-c33c-4f0e-bce8-4f9098fb2d5c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.124865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.126007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.126293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.126582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.127216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.129369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.186548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1054/2115] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:04 2022] PUT /identity/v3/projects/eca8c248eefe4c78b78a3eaf4899cb0c/users/ea60e85bf5304df69c99ed1ae98c0ef3/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:04.197333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.197623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.197902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.225700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] MFA Rules not processed for user `ea60e85bf5304df69c99ed1ae98c0ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:04.226896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:04.270572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1062/2116] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:04.271773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.312217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1055/2117] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:04.328388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.329830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:04.352052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aef9b4-3d13-4125-8c12-8ace6226ba7b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.357816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.358051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.358307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.358586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.359523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.361251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.368317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:04.391817 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c5fd-2864-4d4b-9423-042a6d03020c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.444205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.444831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.445111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.445383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.465293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1063/2118] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:04.469779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.480509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841040a-b36c-4556-a2fd-f599b3ccf2be admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.485518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.486244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.486628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.486998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.487746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.489736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.504507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1064/2119] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:04.507237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.507912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.509407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.519850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1056/2120] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:04.525725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c492c9-4256-4951-bbe0-8838896ee7fa admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.533164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.534059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.534341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.534706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.535256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.541469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.542141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.554230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb16d3-6047-435f-9f33-8700a035162e service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.554452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1065/2121] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:04.560842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.561893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.562028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.562135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.562410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] PATH_INFO: `/v3/users/ea60e85bf5304df69c99ed1ae98c0ef3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.565332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea60e85bf5304df69c99ed1ae98c0ef3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.567365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.572550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043131d-bd1f-4a66-aa47-42f324557c31 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:04.577792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f515990-8f56-49ec-b929-7e240809f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:04.578585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f515990-8f56-49ec-b929-7e240809f7dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.578977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f515990-8f56-49ec-b929-7e240809f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.579325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f515990-8f56-49ec-b929-7e240809f7dc admin admin] PATH_INFO: `/v3/projects/1cbf5a27a5204c8ca19b140508f8378b/users/83f0316f809747c0935e3fe2265e0bf4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.606802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f515990-8f56-49ec-b929-7e240809f7dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbf5a27a5204c8ca19b140508f8378b, user_id=83f0316f809747c0935e3fe2265e0bf4, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:04.609662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f515990-8f56-49ec-b929-7e240809f7dc admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:04.706130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1066/2122] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:04 2022] PUT /identity/v3/projects/1cbf5a27a5204c8ca19b140508f8378b/users/83f0316f809747c0935e3fe2265e0bf4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:04.709550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.709941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.710334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.745949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] MFA Rules not processed for user `83f0316f809747c0935e3fe2265e0bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:04.747518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:04.770099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea60e85bf5304df69c99ed1ae98c0ef3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:04.771724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea60e85bf5304df69c99ed1ae98c0ef3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:04.772444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea60e85bf5304df69c99ed1ae98c0ef3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:04.784334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:04.842303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea60e85bf5304df69c99ed1ae98c0ef3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:04.852232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c0cd420-bdcb-4f24-bb99-3e4936733cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea60e85bf5304df69c99ed1ae98c0ef3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:04.852451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1067/2123] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:04.856759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:04.860697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:04.903347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1057/2124] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:04 2022] DELETE /identity/v3/users/ea60e85bf5304df69c99ed1ae98c0ef3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:04.908900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:04.909186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:04.910113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:04.939028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:04.940024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:05.010546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1058/2125] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:05.017873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c3df6-e43e-456f-a34b-99ed92c7306d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.021647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.023928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:05.025311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.026417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.026899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.027373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.028084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.030229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.097046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f1cb-b70b-4161-a4a6-ae7a3e1ad6b5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.106835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.106835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.106835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.106835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.116021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.118088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.131463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1059/2126] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:05.141984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1068/2127] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:04 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:05.143484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.166725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dd2a4-172c-4f59-8cb7-c5e0902deb46 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.278929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.279747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.285838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.286023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.286304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.287238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.288009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:05.289209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.309602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28f96-8483-49f8-a79f-46e710884e02 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.317856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.319290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.319994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.320276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] PATH_INFO: `/v3/users/acedbdbf7c6743788262af5168274991` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.321593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=acedbdbf7c6743788262af5168274991)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.323903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.417083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1060/2128] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:05 2022] POST /identity/v3/users => generated 298 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:05.421052 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.441179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f2b37-465f-484c-ba89-857b04d18611 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.450785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.451452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.451729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.452112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.453323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.455125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.460863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:05.483275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1061/2129] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:05.489939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.500123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cccbf7-0b02-4f08-9e63-194e6bd98cd6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.505264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.506462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.506738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.507005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] PATH_INFO: `/v3/projects/2de5c05a3c274ca782012a67b67cd2b5/users/9be084881f6a46beacfe44fb5572e1d3/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.524238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de5c05a3c274ca782012a67b67cd2b5, user_id=9be084881f6a46beacfe44fb5572e1d3, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.526995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.533098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:05.565512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acedbdbf7c6743788262af5168274991'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:05.613815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1062/2130] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:05 2022] PUT /identity/v3/projects/2de5c05a3c274ca782012a67b67cd2b5/users/9be084881f6a46beacfe44fb5572e1d3/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:05.614921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.687392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acedbdbf7c6743788262af5168274991'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:05.689094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da267df9-b90c-41c2-8b44-af3cef433cbf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.695618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.696270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.698521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.698521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.698799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acedbdbf7c6743788262af5168274991'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:05.698890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.698973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acedbdbf7c6743788262af5168274991'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:05.698973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-eb754738-9265-4f3a-ae18-3daa3e3b5e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acedbdbf7c6743788262af5168274991'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:05.699213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.710679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1063/2131] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:05.714036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.716068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:05.726245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1069/2132] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:05 2022] DELETE /identity/v3/users/acedbdbf7c6743788262af5168274991 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:05.727581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ab037-4617-482c-9649-36691620c25a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.727872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ab037-4617-482c-9649-36691620c25a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.728236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ab037-4617-482c-9649-36691620c25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.734953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39161a-9318-4f2a-8907-f37b4a3607a6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:05.755250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-730ab037-4617-482c-9649-36691620c25a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:05.757508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-730ab037-4617-482c-9649-36691620c25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:05.839526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcd5eb-0229-4b1c-8d91-a3701a733a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:05.840444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcd5eb-0229-4b1c-8d91-a3701a733a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.840783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcd5eb-0229-4b1c-8d91-a3701a733a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.841283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcd5eb-0229-4b1c-8d91-a3701a733a92 admin admin] PATH_INFO: `/v3/projects/2de5c05a3c274ca782012a67b67cd2b5/users/9be084881f6a46beacfe44fb5572e1d3/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.845694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcd5eb-0229-4b1c-8d91-a3701a733a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de5c05a3c274ca782012a67b67cd2b5, user_id=9be084881f6a46beacfe44fb5572e1d3, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:05.848698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcd5eb-0229-4b1c-8d91-a3701a733a92 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:05.880230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1064/2133] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:05 2022] PUT /identity/v3/projects/2de5c05a3c274ca782012a67b67cd2b5/users/9be084881f6a46beacfe44fb5572e1d3/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:05.885522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1070/2134] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:05.891771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:05.892301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:05.892709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:05.914583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:05.933331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] MFA Rules not processed for user `9be084881f6a46beacfe44fb5572e1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:05.934738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:05.951507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ab037-4617-482c-9649-36691620c25a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:05.954142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-730ab037-4617-482c-9649-36691620c25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:06.066124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ab037-4617-482c-9649-36691620c25a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:06.090977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1065/2135] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:06.095386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:06.096808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:06.150320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:06.151323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.151900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.152462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.215239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c76a63-8235-47b5-bc55-88e1f9126125 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:06.222074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:06.223132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.224941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.225713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] PATH_INFO: `/v3/projects/eca8c248eefe4c78b78a3eaf4899cb0c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.227400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca8c248eefe4c78b78a3eaf4899cb0c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:06.229645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:06.263624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:06.265535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:06.277371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1071/2136] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:06.277807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:06.279155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:06.359000 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8764-376c-4bfc-8b7b-18a5a6dc2d6d service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:06.364638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:06.366006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.366344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.366701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.368737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:06.401418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca8c248eefe4c78b78a3eaf4899cb0c', 'request_id': 'req-22d9d847-174c-4557-af3d-84fa7e67c775'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:06.455384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:06.457043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b8189-f1e6-4c6f-96b3-58042ae3de6d service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:06.469372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1072/2137] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:06.482269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca8c248eefe4c78b78a3eaf4899cb0c', 'request_id': 'req-22d9d847-174c-4557-af3d-84fa7e67c775'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:06.488849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca8c248eefe4c78b78a3eaf4899cb0c', 'request_id': 'req-22d9d847-174c-4557-af3d-84fa7e67c775'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:06.501382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] The token cache is being invalidate because project eca8c248eefe4c78b78a3eaf4899cb0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:06.501761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:06.502574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-22d9d847-174c-4557-af3d-84fa7e67c775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:06.507388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1066/2138] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:06 2022] DELETE /identity/v3/projects/eca8c248eefe4c78b78a3eaf4899cb0c => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:06.509354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.509724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.510086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.531613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:06.532941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:06.608229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1073/2139] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:06.616740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.617245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.617533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.625023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:06.659186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:06.660473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:06.683912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1067/2140] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:06.688215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:06.702737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70310698-3aa8-4ec8-9a74-50e9c779737d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:06.716930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:06.717480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.717762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.718037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.718698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:06.719005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:06.720999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:06.752004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1074/2141] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:47:06.756379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:06.770396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429eb8c7-3a12-4054-bee7-66f76b5fae29 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:06.787592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:06.788652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:06.789013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:06.789291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:06.789968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:06.794221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:06.920122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1068/2142] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:06 2022] POST /identity/v3/projects => generated 359 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:06.924800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:06.937458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd31d8-4e84-4d62-b69d-807bfc69edaf admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.040743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.041469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.041822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.042548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.042864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.044909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.075902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.077926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.100114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6490-7a44-45e2-916a-66dbd740a17a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.108083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.108999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.109463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.109926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] PATH_INFO: `/v3/projects/dcbe60d2c893418aaba65a6fc3ed3776` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.111403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbe60d2c893418aaba65a6fc3ed3776)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.113664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.157166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1075/2143] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:06 2022] POST /identity/v3/users => generated 280 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:07.161329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.171804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115233a1-2caf-450e-96b5-bbe15b1205cb admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.178171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.178886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.179242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.179598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.180364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.181984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.194321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1076/2144] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:07.197888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.210087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87dcc-f4a7-4a41-ba28-e517ca04618f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.216122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.217072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.217423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.217782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.218494 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.220184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.232486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1077/2145] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:07.236900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.256481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779f647-15a5-415c-8692-4d2c92beb0b1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.266355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:07.279238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbe60d2c893418aaba65a6fc3ed3776', 'request_id': 'req-d27f042c-2238-4154-ab3f-64d79b69187a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.330355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13ce27-0918-4f24-a4b6-0ae062d83322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.331016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13ce27-0918-4f24-a4b6-0ae062d83322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.331324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13ce27-0918-4f24-a4b6-0ae062d83322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.331724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13ce27-0918-4f24-a4b6-0ae062d83322 admin admin] PATH_INFO: `/v3/projects/7b6b972dc40c465588dff04ecc5d6b3d/users/0abe27a136c64487bcf6991461950d0e/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.347594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13ce27-0918-4f24-a4b6-0ae062d83322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6b972dc40c465588dff04ecc5d6b3d, user_id=0abe27a136c64487bcf6991461950d0e, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.350568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13ce27-0918-4f24-a4b6-0ae062d83322 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.356974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbe60d2c893418aaba65a6fc3ed3776', 'request_id': 'req-d27f042c-2238-4154-ab3f-64d79b69187a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.363076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbe60d2c893418aaba65a6fc3ed3776', 'request_id': 'req-d27f042c-2238-4154-ab3f-64d79b69187a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.395680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] The token cache is being invalidate because project dcbe60d2c893418aaba65a6fc3ed3776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:07.396197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.396903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.398713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1069/2146] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:07 2022] DELETE /identity/v3/projects/dcbe60d2c893418aaba65a6fc3ed3776 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:07.402766 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.404706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.427737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f042c-2238-4154-ab3f-64d79b69187a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.433797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.435358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.435797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.436251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] PATH_INFO: `/v3/users/78fcb8bde16d4d0d92cc1e47b7eaa8f8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.437604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78fcb8bde16d4d0d92cc1e47b7eaa8f8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.439881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.479569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1078/2147] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:07 2022] PUT /identity/v3/projects/7b6b972dc40c465588dff04ecc5d6b3d/users/0abe27a136c64487bcf6991461950d0e/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:07.484882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.487072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.489611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.550621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] MFA Rules not processed for user `0abe27a136c64487bcf6991461950d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:07.557320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.628323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1079/2148] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:07.669425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78fcb8bde16d4d0d92cc1e47b7eaa8f8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.670226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78fcb8bde16d4d0d92cc1e47b7eaa8f8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.670848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78fcb8bde16d4d0d92cc1e47b7eaa8f8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.671519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.673052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.730592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658439b-a24f-4ab2-9d17-9b7268919664 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.734618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.735218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.735492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.735759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.737583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.748238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.749721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.758984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1080/2149] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:47:07.761702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78fcb8bde16d4d0d92cc1e47b7eaa8f8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.770750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-62372278-6285-4c60-8429-51c70e65b835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78fcb8bde16d4d0d92cc1e47b7eaa8f8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:07.777128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:07.778609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.792892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1070/2150] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:07 2022] DELETE /identity/v3/users/78fcb8bde16d4d0d92cc1e47b7eaa8f8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:07.842593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f84ab-f6d0-42fb-b808-96705982a487 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.843567 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f84ab-f6d0-42fb-b808-96705982a487 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.844165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f84ab-f6d0-42fb-b808-96705982a487 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.848289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1071/2151] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:47:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:47:07.854504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e176-6f63-4091-b4a4-a3e73d3592d5 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:07.883888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33896d08-812d-4d17-81b7-fa6a08aa8dac None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.884463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33896d08-812d-4d17-81b7-fa6a08aa8dac None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.884840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33896d08-812d-4d17-81b7-fa6a08aa8dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.909383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-33896d08-812d-4d17-81b7-fa6a08aa8dac None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:07.910599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-33896d08-812d-4d17-81b7-fa6a08aa8dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:07.917613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:07.918310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.918588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.918864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] PATH_INFO: `/v3/users/0abe27a136c64487bcf6991461950d0e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.920008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0abe27a136c64487bcf6991461950d0e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:07.921974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:07.975628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1072/2152] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:47:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:07.986651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a7fd-a0c2-424c-a133-d5bbd8038524 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.986986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a7fd-a0c2-424c-a133-d5bbd8038524 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.987210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171a7fd-a0c2-424c-a133-d5bbd8038524 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:07.988529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1073/2153] 10.222.0.57 () {54 vars in 961 bytes} [Wed Sep 7 14:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 14:47:07.993868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:07.994150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:07.994428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.016928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] MFA Rules not processed for user `53d54b01ab3c4003b5f13e20c0849db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:08.018241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.032536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1074/2154] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Sep 7 14:47:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:08.033161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:08.034557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.051550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d641f38-883a-42fc-a10c-6e9790272f21 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:08.051775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:08.057314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:08.057953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.058233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.058503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] PATH_INFO: `/v3/users/9f46e69140e347669da323ad08edd7e5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.059687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f46e69140e347669da323ad08edd7e5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:08.061744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:08.176881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0abe27a136c64487bcf6991461950d0e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.267924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f46e69140e347669da323ad08edd7e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.268560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f46e69140e347669da323ad08edd7e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.269165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f46e69140e347669da323ad08edd7e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.276455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0abe27a136c64487bcf6991461950d0e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.276716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:08.283941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0abe27a136c64487bcf6991461950d0e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.284614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0abe27a136c64487bcf6991461950d0e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.285308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0abe27a136c64487bcf6991461950d0e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.303176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1081/2155] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:07 2022] DELETE /identity/v3/users/0abe27a136c64487bcf6991461950d0e => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:08.304004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:08.305595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.361865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f46e69140e347669da323ad08edd7e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.373530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-547edae6-1640-435a-8fa4-a4433ff50be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f46e69140e347669da323ad08edd7e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.381446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73722eb-67bf-41fc-a73c-09ff9bdf5c62 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:08.386633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:08.388023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.388023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.388023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.389658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.397093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1075/2156] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:08 2022] DELETE /identity/v3/users/9f46e69140e347669da323ad08edd7e5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:08.398687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.399047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.399955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.402047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:08.404042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:08.414226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1082/2157] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:08.415053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:08.416430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.438184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:08.442403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.532948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925429ab-0236-4334-875e-6c34d85b31d7 service placement] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:08.539556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:08.544698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.545311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1076/2158] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:08.545478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.545478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] PATH_INFO: `/v3/projects/b6fbaaf583bc44f88dc5d08ed1b65894` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.545478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fbaaf583bc44f88dc5d08ed1b65894)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:08.545478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:08.555778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:08.556936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.652332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc36a08-7d5f-4d71-8632-e6cd7283e78e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:08.657433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:08.658197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.659192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.659536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.671389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:08.673068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:08.690803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1077/2159] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:08.714026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fbaaf583bc44f88dc5d08ed1b65894', 'request_id': 'req-b20dcfac-7cd8-4854-a388-519c90d8468e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.722146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:08.788181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:08.790076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.800563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fbaaf583bc44f88dc5d08ed1b65894', 'request_id': 'req-b20dcfac-7cd8-4854-a388-519c90d8468e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.805152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4ee43-8a71-42ca-837a-4efee9a78c2e service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:08.808878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fbaaf583bc44f88dc5d08ed1b65894', 'request_id': 'req-b20dcfac-7cd8-4854-a388-519c90d8468e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.825850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] The token cache is being invalidate because project b6fbaaf583bc44f88dc5d08ed1b65894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:08.826678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.826861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:08.828694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1083/2160] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:08 2022] DELETE /identity/v3/projects/b6fbaaf583bc44f88dc5d08ed1b65894 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:08.832296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:08.834490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:08.885198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:08.885896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.886180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.886453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] PATH_INFO: `/v3/projects/d95dab0bcea84d42b8b92ce81606e2ec` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.887598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95dab0bcea84d42b8b92ce81606e2ec)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:08.889775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:08.923024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dcfac-7cd8-4854-a388-519c90d8468e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:08.928103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:08.928803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:08.929056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:08.929336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] PATH_INFO: `/v3/users/5372d4a4b0af4ccc92743b8d3178c2c5` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:08.930483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5372d4a4b0af4ccc92743b8d3178c2c5)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:08.932469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.079815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95dab0bcea84d42b8b92ce81606e2ec', 'request_id': 'req-1c7077e2-db0a-49df-9b07-e158cda76df7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.195782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95dab0bcea84d42b8b92ce81606e2ec', 'request_id': 'req-1c7077e2-db0a-49df-9b07-e158cda76df7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.201942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95dab0bcea84d42b8b92ce81606e2ec', 'request_id': 'req-1c7077e2-db0a-49df-9b07-e158cda76df7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.213941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] The token cache is being invalidate because project d95dab0bcea84d42b8b92ce81606e2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:09.214233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.214902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.216435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1078/2161] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:08 2022] DELETE /identity/v3/projects/d95dab0bcea84d42b8b92ce81606e2ec => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:09.217305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.218592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.242692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5372d4a4b0af4ccc92743b8d3178c2c5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.312636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7077e2-db0a-49df-9b07-e158cda76df7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.318366 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.319082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.319423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.319700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] PATH_INFO: `/v3/projects/7b6b972dc40c465588dff04ecc5d6b3d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.320851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6b972dc40c465588dff04ecc5d6b3d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.322944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.338997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5372d4a4b0af4ccc92743b8d3178c2c5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.347602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5372d4a4b0af4ccc92743b8d3178c2c5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.349712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5372d4a4b0af4ccc92743b8d3178c2c5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.353854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-19b8d8bc-b9c7-4881-b64c-cb4d48ba5465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5372d4a4b0af4ccc92743b8d3178c2c5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.353972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:09.374628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1084/2162] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:08 2022] DELETE /identity/v3/users/5372d4a4b0af4ccc92743b8d3178c2c5 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:09.374628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93c10-f2ca-474a-a678-97ffe575871b None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.374628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93c10-f2ca-474a-a678-97ffe575871b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.374628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93c10-f2ca-474a-a678-97ffe575871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.409262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-26a93c10-f2ca-474a-a678-97ffe575871b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:09.410497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-26a93c10-f2ca-474a-a678-97ffe575871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.456367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6b972dc40c465588dff04ecc5d6b3d', 'request_id': 'req-af0c4189-d37b-4d0f-b421-4618fc299aeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.476896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1085/2163] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:09.484634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.484919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.485187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.508962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:09.510183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.556616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6b972dc40c465588dff04ecc5d6b3d', 'request_id': 'req-af0c4189-d37b-4d0f-b421-4618fc299aeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.563272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6b972dc40c465588dff04ecc5d6b3d', 'request_id': 'req-af0c4189-d37b-4d0f-b421-4618fc299aeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.574166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1086/2164] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:09.579254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] The token cache is being invalidate because project 7b6b972dc40c465588dff04ecc5d6b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:09.579563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.580284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-af0c4189-d37b-4d0f-b421-4618fc299aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:09.580693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.582367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1079/2165] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:09 2022] DELETE /identity/v3/projects/7b6b972dc40c465588dff04ecc5d6b3d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:09.591606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852f8da-f740-46e3-aa7d-64b5046f9def None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.596814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.597613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.597992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.598360 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.601927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:09.601927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.602109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234f34-fca6-434a-bd1c-21a1de4690eb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.602109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234f34-fca6-434a-bd1c-21a1de4690eb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.602109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234f34-fca6-434a-bd1c-21a1de4690eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.602791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.622081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1087/2166] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:47:09.626137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.626137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.640983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3a234f34-fca6-434a-bd1c-21a1de4690eb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:09.641336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3697bb7-e3c5-4a06-9e61-3e4be1757548 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.642468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3a234f34-fca6-434a-bd1c-21a1de4690eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.648257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.648875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.649155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.649432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.650072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.652224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.654348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1080/2167] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:09.662272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.662559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.662875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.683810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:09.685100 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.693946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1081/2168] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:09.698044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.707168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339bc4c-478f-4c48-a980-0ebce99a9d01 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.712279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.712859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.723792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.723792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.723792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:09.723792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.723792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.727799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1082/2169] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:47:09.731886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.741367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afeafc-c936-4d0b-905b-5b13ca1d631c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.746048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.746610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.746890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.747162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.747780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.749714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.754199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1088/2170] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:09.755327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.820317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29904b7-f7dd-4525-926e-e8259c6b7791 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.824923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.825526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.825808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.826077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.826702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.828661 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.860369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1083/2171] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:09.864130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.907956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1089/2172] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:09.908261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:09.912118 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:09.925952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbf81b-e9d0-40f6-a027-815764cdc0e5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.936963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c923db1-7baa-4351-a5f5-c6268ccb1bdc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:09.942213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.943400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.943597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.943935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.944537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.946638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:09.992243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:09.992823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:09.993105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:09.993377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] PATH_INFO: `/v3/projects/1c80f7f852aa4658b06a8469a350fb31` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:09.994578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c80f7f852aa4658b06a8469a350fb31)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:09.996539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.024483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1084/2173] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:09 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:10.025257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.034997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace75a2-4bd0-4b0c-8d1d-99e022173765 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.039856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.040391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.040667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.040934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.041576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.043171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.054025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1085/2174] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:10.054861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.066279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fc895-190d-4ace-af86-ead40d36af54 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.071566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.072087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.072401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.072672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.073295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.074863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.085964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1086/2175] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:10.086759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.096134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167fa5d-0832-46a8-8a12-27c992907ca6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.100791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.101367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.101647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.103508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.103508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.104084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.113923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1087/2176] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:10.114536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.118866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c80f7f852aa4658b06a8469a350fb31', 'request_id': 'req-b41d86ba-bb35-4e47-9159-878749663805'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:10.124291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16579648-7360-48d7-be1e-e7027a61609e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.179734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.180367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.180644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.180914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.181548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.183204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.192760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1088/2177] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:10.194116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.195047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c80f7f852aa4658b06a8469a350fb31', 'request_id': 'req-b41d86ba-bb35-4e47-9159-878749663805'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:10.201315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c80f7f852aa4658b06a8469a350fb31', 'request_id': 'req-b41d86ba-bb35-4e47-9159-878749663805'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:10.204886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e686a6d-88ad-4a6c-92fb-c69827a3b4c1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.210411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9ea66-5f6a-4254-a104-3529af6f4554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.211887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9ea66-5f6a-4254-a104-3529af6f4554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.211887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9ea66-5f6a-4254-a104-3529af6f4554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.211887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9ea66-5f6a-4254-a104-3529af6f4554 admin admin] PATH_INFO: `/v3/projects/3604599543654dcaab3ae36a890c065c/users/7aaa2615f85c417d97c8d6a2c6430768/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.213943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] The token cache is being invalidate because project 1c80f7f852aa4658b06a8469a350fb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:10.214235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:10.214908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:10.216750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1090/2178] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:09 2022] DELETE /identity/v3/projects/1c80f7f852aa4658b06a8469a350fb31 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:10.221163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.222876 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:10.228365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9ea66-5f6a-4254-a104-3529af6f4554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3604599543654dcaab3ae36a890c065c, user_id=7aaa2615f85c417d97c8d6a2c6430768, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.233178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9ea66-5f6a-4254-a104-3529af6f4554 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.235519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d86ba-bb35-4e47-9159-878749663805 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.241060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6252f6-8439-45ac-83de-8595e219b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.241585 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6252f6-8439-45ac-83de-8595e219b38c admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.241861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6252f6-8439-45ac-83de-8595e219b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.242174 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6252f6-8439-45ac-83de-8595e219b38c admin admin] PATH_INFO: `/v3/projects/8ce9624faef1455786800a976f6f2722/users/da647e7722234bcaad49c106ac3abba4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.257775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6252f6-8439-45ac-83de-8595e219b38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce9624faef1455786800a976f6f2722, user_id=da647e7722234bcaad49c106ac3abba4, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.260474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6252f6-8439-45ac-83de-8595e219b38c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.334449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1089/2179] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:10 2022] PUT /identity/v3/projects/3604599543654dcaab3ae36a890c065c/users/7aaa2615f85c417d97c8d6a2c6430768/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:10.336014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d248-3fd3-42f4-8c57-433b71471939 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.336329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d248-3fd3-42f4-8c57-433b71471939 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.336599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d248-3fd3-42f4-8c57-433b71471939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.346167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:10.352359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1091/2180] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:10 2022] PUT /identity/v3/projects/8ce9624faef1455786800a976f6f2722/users/da647e7722234bcaad49c106ac3abba4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:10.353872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879674b-a6ec-42fb-898c-086d5c375dde None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.354670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879674b-a6ec-42fb-898c-086d5c375dde None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.355286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879674b-a6ec-42fb-898c-086d5c375dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.390750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d879674b-a6ec-42fb-898c-086d5c375dde None None] MFA Rules not processed for user `7aaa2615f85c417d97c8d6a2c6430768`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:10.390885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a926d248-3fd3-42f4-8c57-433b71471939 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:10.390980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d879674b-a6ec-42fb-898c-086d5c375dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:10.392033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a926d248-3fd3-42f4-8c57-433b71471939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:10.463361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1090/2181] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:10.464821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.465186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.465549 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.498328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] MFA Rules not processed for user `da647e7722234bcaad49c106ac3abba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:10.499673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:10.535673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1092/2182] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:10.540686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.540886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.541312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.564586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:10.565974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:10.579603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1093/2183] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:10.579603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.579603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:10.613421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1091/2184] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:10.616491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.638804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bee28-f91f-428f-b5bb-cfd9bcfadde7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.647624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.648720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.649163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.649433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.652046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:10.652384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.652982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:10.656604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.657739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfecbe6-cc1a-45bd-9f52-94848db1234a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.661963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.662766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.663200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.663777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.674152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1092/2185] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:47:10.675033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.676462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.677965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.685459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7d706-d569-418d-9aca-8655d674f61c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.688626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1094/2186] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:10.689415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.690580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.691179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.691512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.691955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.692604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.695156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.701689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301ab61-cb69-46ab-bd4e-1255487dee88 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.707793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.709605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.709605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.716508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.716508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.716508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.828181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1093/2187] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:10.832994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.838433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1095/2188] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:10.842813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:10.904524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a532-0bda-4a56-8e95-9a1ac37ab388 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.924168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a403a-bae6-4c42-a48b-e522e19f2143 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:10.929039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.929641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.929921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.930193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.930821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.932799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:10.967426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:10.971649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:10.973748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:10.974060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:10.974850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:10.975144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:10.977310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.004178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1096/2189] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:11.007140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.017238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5655bc-cc38-4a8a-b320-6f7ac7f6c650 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.022201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.022849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.023216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.023392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.024069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.025722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.036006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1097/2190] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:11.042066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.051977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5678c-9241-4580-ad0d-f50f089bc557 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.057909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.058539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.058868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.059149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.059796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.061492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.077177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1098/2191] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:11.078061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.093563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66971-0452-448c-a313-72a58ecaba24 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.094678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1094/2192] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:10 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:11.095792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.099075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fdb32-cff6-4175-b309-3830be240775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.099746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fdb32-cff6-4175-b309-3830be240775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.100055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fdb32-cff6-4175-b309-3830be240775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.100621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fdb32-cff6-4175-b309-3830be240775 admin admin] PATH_INFO: `/v3/projects/d92aee0ad4664de698592bf4cc20b5d6/users/9b044e9de07e4a1c80f8fbb94c66ca55/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.117727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012826ff-ff1a-43c9-9857-a2a7c5595e14 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.121209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fdb32-cff6-4175-b309-3830be240775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92aee0ad4664de698592bf4cc20b5d6, user_id=9b044e9de07e4a1c80f8fbb94c66ca55, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.122723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.123393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.123674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.124644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.124757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fdb32-cff6-4175-b309-3830be240775 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.124932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.126470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.138015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1095/2193] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:11.142339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.153065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7a95-ede8-4b54-ab13-23d9814a3ede admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.158842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.159453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.159738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.160035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.160705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.162358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.173660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1096/2194] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:11.181240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.193511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0890a6-f2dd-467d-866d-1c92f50ded37 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.198284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.199011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.199349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.199666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] PATH_INFO: `/v3/projects/5d25cb6decce42abb2ee47d69325643f/users/0f1b7f9296cf4d6fb9f5b061bd795cfa/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.212490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1099/2195] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:11 2022] PUT /identity/v3/projects/d92aee0ad4664de698592bf4cc20b5d6/users/9b044e9de07e4a1c80f8fbb94c66ca55/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:11.218575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91a5a2-2a5b-40a7-9281-8a861a427453 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.218866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91a5a2-2a5b-40a7-9281-8a861a427453 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.219383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91a5a2-2a5b-40a7-9281-8a861a427453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.223673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d25cb6decce42abb2ee47d69325643f, user_id=0f1b7f9296cf4d6fb9f5b061bd795cfa, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.226487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.251110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2b91a5a2-2a5b-40a7-9281-8a861a427453 None None] MFA Rules not processed for user `9b044e9de07e4a1c80f8fbb94c66ca55`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:11.253222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2b91a5a2-2a5b-40a7-9281-8a861a427453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:11.335751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1100/2196] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:11.335919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1097/2197] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:11 2022] PUT /identity/v3/projects/5d25cb6decce42abb2ee47d69325643f/users/0f1b7f9296cf4d6fb9f5b061bd795cfa/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:11.344844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.345161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.345435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.350724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.352449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:11.374968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] MFA Rules not processed for user `0f1b7f9296cf4d6fb9f5b061bd795cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:11.376270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:11.416216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709bc51-ca49-4024-86a7-52e26afd2931 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.420755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.421456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.421735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.422065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.456418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1101/2198] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:11.461131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.489203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.496447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.505500 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1098/2199] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:47:11.529014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d111c-d3a1-4d27-8ed1-1d6df73f5b05 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.533702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.534320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.534593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.534855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.535501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.538025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.621416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1102/2200] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:11 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:11.625471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.637564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99625b9-b4e7-4b1f-b228-0df5b86f343b service glance] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.721890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.726132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.726132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.726132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.726132 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.726573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.848759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1099/2201] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:11 2022] POST /identity/v3/users => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:11.852621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.863062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8e96-4992-4812-ba4b-8579003f764d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.867675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.868318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.874656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.874656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.874656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.874656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:11.881476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1103/2202] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:11.886093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:11.887736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:11.896784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51915b00-f413-412c-ae11-984d097bb69e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:11.901418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:11.902065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:11.902412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:11.902616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] PATH_INFO: `/v3/projects/8816be837ab444d0aeaa5bb8a0695e89/users/63353800165546f4971e83489a2a9cdc/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:11.918521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8816be837ab444d0aeaa5bb8a0695e89, user_id=63353800165546f4971e83489a2a9cdc, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:11.921281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:12.042998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1100/2203] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:11 2022] PUT /identity/v3/projects/8816be837ab444d0aeaa5bb8a0695e89/users/63353800165546f4971e83489a2a9cdc/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:12.047199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:12.057203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9f8fa-e03b-4df7-a3b6-168109d0b285 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:12.113149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af867b1-aa7f-45f4-8e08-d05cdc80e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:12.113685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af867b1-aa7f-45f4-8e08-d05cdc80e1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:12.113967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af867b1-aa7f-45f4-8e08-d05cdc80e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:12.114239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af867b1-aa7f-45f4-8e08-d05cdc80e1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:12.114900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af867b1-aa7f-45f4-8e08-d05cdc80e1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:12.116725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af867b1-aa7f-45f4-8e08-d05cdc80e1b0 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:12.126411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1104/2204] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:12.131363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:12.142303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77150d8b-4816-4e76-99f6-17f4d018c9e0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:12.147727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:12.148443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:12.148701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:12.149023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] PATH_INFO: `/v3/projects/8816be837ab444d0aeaa5bb8a0695e89/users/63353800165546f4971e83489a2a9cdc/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:12.150885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8816be837ab444d0aeaa5bb8a0695e89, user_id=63353800165546f4971e83489a2a9cdc, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:12.153581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:12.164016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1101/2205] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:12 2022] PUT /identity/v3/projects/8816be837ab444d0aeaa5bb8a0695e89/users/63353800165546f4971e83489a2a9cdc/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:12.174253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:12.174542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:12.174821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:12.204493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] MFA Rules not processed for user `63353800165546f4971e83489a2a9cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:12.205813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:12.283418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1105/2206] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:12.301837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:12.304575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:12.323262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:12.349814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:12.383162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5e5d-29a4-4e7c-b65e-8d5858cd1475 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:12.387428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:12.388498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:12.388918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:12.389359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:12.423228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d1706-0553-425a-83b1-3784f9d031eb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:12.428649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:12.429565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:12.430445 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:12.431025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:12.433749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:12.436938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:12.478713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:12.480508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:12.499658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1102/2207] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:12.539377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:12.541132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:12.551138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1106/2208] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:13.041481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:13.122329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:13.853956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:13.952385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:14.665337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:14.671095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:14.680976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8605b7-3334-4c9c-91a7-236e8030415b service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:14.684593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:14.685223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:14.685501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:14.685851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:14.743181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:14.744676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:14.746351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:14.755374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1103/2209] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:15.241028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:15.316288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:15.910819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:16.288014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:16.444589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:16.453235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b46003-e7c7-41a9-80f4-cd8207664e18 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:16.457194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652aab3c-54a6-4b77-8ce0-f0c71abbc303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:16.457764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652aab3c-54a6-4b77-8ce0-f0c71abbc303 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:16.460420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652aab3c-54a6-4b77-8ce0-f0c71abbc303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:16.463461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652aab3c-54a6-4b77-8ce0-f0c71abbc303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:16.470905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652aab3c-54a6-4b77-8ce0-f0c71abbc303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:16.472397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652aab3c-54a6-4b77-8ce0-f0c71abbc303 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:16.483618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1107/2210] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:18.780698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:18.782279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:18.860127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af435fc-936d-409b-973a-ec3ce0b7c91a service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:18.866432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:18.867225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:18.867502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:18.867774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] PATH_INFO: `/v3/users/83f0316f809747c0935e3fe2265e0bf4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:18.871380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f0316f809747c0935e3fe2265e0bf4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:18.871380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:19.083542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f0316f809747c0935e3fe2265e0bf4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:19.084539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f0316f809747c0935e3fe2265e0bf4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:19.085863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f0316f809747c0935e3fe2265e0bf4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:19.088716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:19.165881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f0316f809747c0935e3fe2265e0bf4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:19.174872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f0316f809747c0935e3fe2265e0bf4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:19.197486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1104/2211] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:18 2022] DELETE /identity/v3/users/83f0316f809747c0935e3fe2265e0bf4 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:19.201974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:19.202464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:19.202619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:19.221978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:19.223196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:19.284895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1108/2212] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:19.295321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:19.297101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:19.368796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3299f-a3c8-4948-867c-b0fd5311123c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:19.378120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:19.379991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:19.380957 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:19.381412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:19.403477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:19.417336 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:19.453737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1105/2213] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:20.054483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:20.055975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:20.073655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ce543-4052-4b83-a233-ffe23f698ee8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:20.078387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:20.079352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:20.079909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:20.080244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] PATH_INFO: `/v3/projects/1cbf5a27a5204c8ca19b140508f8378b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:20.081424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbf5a27a5204c8ca19b140508f8378b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:20.083376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:20.193801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbf5a27a5204c8ca19b140508f8378b', 'request_id': 'req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.241444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbf5a27a5204c8ca19b140508f8378b', 'request_id': 'req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.264173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbf5a27a5204c8ca19b140508f8378b', 'request_id': 'req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.267960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] The token cache is being invalidate because project 1cbf5a27a5204c8ca19b140508f8378b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:20.267960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.268792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.270975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1109/2214] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:20 2022] DELETE /identity/v3/projects/1cbf5a27a5204c8ca19b140508f8378b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:20.280274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:20.290857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7710-3ebd-4fb1-8542-0ccb94923817 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:20.356098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:20.356775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:20.357055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:20.357328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] PATH_INFO: `/v3/users/9be084881f6a46beacfe44fb5572e1d3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:20.358489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be084881f6a46beacfe44fb5572e1d3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:20.360477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:20.563993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be084881f6a46beacfe44fb5572e1d3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.564669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be084881f6a46beacfe44fb5572e1d3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.565308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be084881f6a46beacfe44fb5572e1d3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.630060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be084881f6a46beacfe44fb5572e1d3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.639701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be084881f6a46beacfe44fb5572e1d3'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:20.648407 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:20.659508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1106/2215] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:20 2022] DELETE /identity/v3/users/9be084881f6a46beacfe44fb5572e1d3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:21.332452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:21.334798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:21.360733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a7b00-8f96-4bda-bd04-8e8045ee5b5a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:21.417954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:21.440583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:21.440583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.440583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.440583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] PATH_INFO: `/v3/projects/2de5c05a3c274ca782012a67b67cd2b5` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.442047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de5c05a3c274ca782012a67b67cd2b5)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:21.446144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:21.447187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:21.456887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:21.472649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a01f15-0bbb-4089-8938-ec73d6e08f69 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:21.480276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:21.481063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.481469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.482209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] PATH_INFO: `/v3/users/da647e7722234bcaad49c106ac3abba4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.483153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=da647e7722234bcaad49c106ac3abba4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:21.485331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:21.557314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de5c05a3c274ca782012a67b67cd2b5', 'request_id': 'req-9af32624-845b-4c78-a209-b562420257f6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.640630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de5c05a3c274ca782012a67b67cd2b5', 'request_id': 'req-9af32624-845b-4c78-a209-b562420257f6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.647221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de5c05a3c274ca782012a67b67cd2b5', 'request_id': 'req-9af32624-845b-4c78-a209-b562420257f6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.655908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da647e7722234bcaad49c106ac3abba4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.656588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da647e7722234bcaad49c106ac3abba4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.657195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da647e7722234bcaad49c106ac3abba4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.659627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] The token cache is being invalidate because project 2de5c05a3c274ca782012a67b67cd2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:21.659938 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.660617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-9af32624-845b-4c78-a209-b562420257f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.662210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1110/2216] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:21 2022] DELETE /identity/v3/projects/2de5c05a3c274ca782012a67b67cd2b5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:21.669977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d546e5-5abe-46f5-9222-0118b000d814 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.670329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d546e5-5abe-46f5-9222-0118b000d814 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.670664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d546e5-5abe-46f5-9222-0118b000d814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.693048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a9d546e5-5abe-46f5-9222-0118b000d814 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:21.694479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a9d546e5-5abe-46f5-9222-0118b000d814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:21.710551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da647e7722234bcaad49c106ac3abba4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.718281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-aa9d0af0-4bb8-4302-b291-5e758943e2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da647e7722234bcaad49c106ac3abba4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:21.736611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1107/2217] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:21 2022] DELETE /identity/v3/users/da647e7722234bcaad49c106ac3abba4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:21.740936 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.741333 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.741696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.759243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1111/2218] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:21.766969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:21.766969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:21.770049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.770517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.770885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.795545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:21.796818 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:21.827703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1108/2219] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:21.841456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:21.841456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:21.859245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1112/2220] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:21.864021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:21.874727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3450005-fe12-4dc0-9da3-97a75352384a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:21.879720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:21.880354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.880632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.880899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.881548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:21.882672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:21.884687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:21.895609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:21.901451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1113/2221] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:47:21.909131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137867-cd00-4c00-a116-b734f1a754e3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:21.909286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:21.912912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:21.913617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.913893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.914160 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.918798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebe7c4-4a45-40b0-a56b-f8789a380ee9 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:21.923227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:21.923547 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:21.924155 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:21.924448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:21.924706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:21.924797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:21.925341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:21.927748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:21.934781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1109/2222] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:22.042974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1114/2223] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:21 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:22.047680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.053330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:22.056972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d0fa6-506f-4328-b183-769112ec6317 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.105482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.107522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.115099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.115715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.115987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.116318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.116954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.118856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.175895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1b17-7e28-41ab-946d-076473c95ebf admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.181248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.182064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.182337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.182605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] PATH_INFO: `/v3/users/7aaa2615f85c417d97c8d6a2c6430768` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.183849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aaa2615f85c417d97c8d6a2c6430768)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.185842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.232207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1110/2224] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:22 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:22.233914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.245493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a88b7-20dc-4523-8002-a4cb61e2bec2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.256183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.256183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.256183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.256183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.256580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.258372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.268945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1111/2225] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:22.272791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.283207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d2654-ca56-4454-8a70-5aeba9c7c9e5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.289721 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.290409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.290688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.291120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.292068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.294332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.308261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1112/2226] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:22.321325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.322590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebb2a-1ef9-4a8d-b40f-b31c15e17de0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.327217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.327851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.328125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.328423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] PATH_INFO: `/v3/projects/b5e50b041ba6416db1001d9b2312f794/users/17b1ae55272b45488d611968e4cdfc76/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.343436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e50b041ba6416db1001d9b2312f794, user_id=17b1ae55272b45488d611968e4cdfc76, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.347308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.404450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aaa2615f85c417d97c8d6a2c6430768'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.454743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1113/2227] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:22 2022] PUT /identity/v3/projects/b5e50b041ba6416db1001d9b2312f794/users/17b1ae55272b45488d611968e4cdfc76/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:22.459000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.460946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.473521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aaa2615f85c417d97c8d6a2c6430768'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.482158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aaa2615f85c417d97c8d6a2c6430768'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.484125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aaa2615f85c417d97c8d6a2c6430768'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.484658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58670c05-0bc4-4f4b-968a-350762c6e69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aaa2615f85c417d97c8d6a2c6430768'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.512854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1115/2228] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:22 2022] DELETE /identity/v3/users/7aaa2615f85c417d97c8d6a2c6430768 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:22.517421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d4a90-f6fc-433f-a94e-73ef05496530 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.517813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d4a90-f6fc-433f-a94e-73ef05496530 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.518192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d4a90-f6fc-433f-a94e-73ef05496530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.550278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-431d4a90-f6fc-433f-a94e-73ef05496530 None None] MFA Rules not processed for user `17b1ae55272b45488d611968e4cdfc76`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:22.551520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-431d4a90-f6fc-433f-a94e-73ef05496530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.570981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7bde-b603-474d-870e-d06f2ca0c12c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.635492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:22.656790 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1116/2229] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:22.662274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.662584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.664214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:22.664365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.676422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.677163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.677507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.677896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] PATH_INFO: `/v3/projects/8ce9624faef1455786800a976f6f2722` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.679743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce9624faef1455786800a976f6f2722)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.681949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.685862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:22.687241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.699355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1117/2230] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:22.703338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.704718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.763296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03430-7731-4738-80d1-3bd0cde7aade None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.768210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce9624faef1455786800a976f6f2722', 'request_id': 'req-ec3092d3-81a1-41e9-8324-866a44566ae2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.817507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.818110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.818383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.818645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.852759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce9624faef1455786800a976f6f2722', 'request_id': 'req-ec3092d3-81a1-41e9-8324-866a44566ae2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.858980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce9624faef1455786800a976f6f2722', 'request_id': 'req-ec3092d3-81a1-41e9-8324-866a44566ae2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.871982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] The token cache is being invalidate because project 8ce9624faef1455786800a976f6f2722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:22.872391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.873099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:22.874597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1114/2231] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:22 2022] DELETE /identity/v3/projects/8ce9624faef1455786800a976f6f2722 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:22.875965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.877555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.882972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.884999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.891168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1118/2232] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:22.891168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:22.891168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.950768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3092d3-81a1-41e9-8324-866a44566ae2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.955414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.956018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.956372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.956649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.957203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:22.959692 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:22.981805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:22.984239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:22.990193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578ebde-8018-4ec4-8ab6-fc1ca1eb5f19 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:22.996180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:22.996940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:22.997319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:22.997666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] PATH_INFO: `/v3/users/0f1b7f9296cf4d6fb9f5b061bd795cfa` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:22.998887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1b7f9296cf4d6fb9f5b061bd795cfa)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:23.000748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1115/2233] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:23.001086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:23.279158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1b7f9296cf4d6fb9f5b061bd795cfa'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.361755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1b7f9296cf4d6fb9f5b061bd795cfa'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.370671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1b7f9296cf4d6fb9f5b061bd795cfa'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.371569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1b7f9296cf4d6fb9f5b061bd795cfa'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.372211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-021469c5-1b3e-4a3d-b8f9-ace60782179a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1b7f9296cf4d6fb9f5b061bd795cfa'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.399120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1119/2234] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:22 2022] DELETE /identity/v3/users/0f1b7f9296cf4d6fb9f5b061bd795cfa => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:23.504879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:23.506363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:23.519895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e99f-6578-4549-bfa3-a3b4762b93b4 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:23.601249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:23.602057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:23.602057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:23.602426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] PATH_INFO: `/v3/projects/3604599543654dcaab3ae36a890c065c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:23.603524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3604599543654dcaab3ae36a890c065c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:23.605455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:23.693474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:23.745645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3604599543654dcaab3ae36a890c065c', 'request_id': 'req-68bc2440-5649-41b0-a418-29036e4b0816'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.877030 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3604599543654dcaab3ae36a890c065c', 'request_id': 'req-68bc2440-5649-41b0-a418-29036e4b0816'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.894265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3604599543654dcaab3ae36a890c065c', 'request_id': 'req-68bc2440-5649-41b0-a418-29036e4b0816'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.917297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] The token cache is being invalidate because project 3604599543654dcaab3ae36a890c065c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:23.917730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.918505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-68bc2440-5649-41b0-a418-29036e4b0816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:23.920553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1116/2235] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:23 2022] DELETE /identity/v3/projects/3604599543654dcaab3ae36a890c065c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:23.925173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:23.925650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:23.925934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:23.950890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:23.952279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.052936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1120/2236] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:24.062649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.063039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.066045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.089221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:24.094888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.095288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.099124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1117/2237] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:24.099258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.103637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.112883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc2d76-ed2b-4ea3-982a-ec07d3666a8b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.120139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec113e5-b84d-4c7e-a4dc-a8d556c1ef67 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.125130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.125770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.126047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.126317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] PATH_INFO: `/v3/projects/5d25cb6decce42abb2ee47d69325643f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.127451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d25cb6decce42abb2ee47d69325643f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.127585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.128463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.128912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.129287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.129432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.130239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:24.130565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.133058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.146869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1118/2238] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:47:24.152159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.161390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6e135-ae0f-438e-9a2a-41b99027bec4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.166666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.167266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.167548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.167824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.168487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.170383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.236862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d25cb6decce42abb2ee47d69325643f', 'request_id': 'req-14ea76a6-09ee-4985-b4da-1724174279dd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.268526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1119/2239] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:24 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:24.270690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.335792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f9b9-154a-4ea7-84f2-f218bba3d011 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.340715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.341327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.341607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.341878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.342507 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.343122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d25cb6decce42abb2ee47d69325643f', 'request_id': 'req-14ea76a6-09ee-4985-b4da-1724174279dd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.344454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.349927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d25cb6decce42abb2ee47d69325643f', 'request_id': 'req-14ea76a6-09ee-4985-b4da-1724174279dd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.351784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:24.372286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] The token cache is being invalidate because project 5d25cb6decce42abb2ee47d69325643f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:24.372663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.373250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.374789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1121/2240] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:24 2022] DELETE /identity/v3/projects/5d25cb6decce42abb2ee47d69325643f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:24.378631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.381523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.445905 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea76a6-09ee-4985-b4da-1724174279dd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.450567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.451298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.451584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.451967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] PATH_INFO: `/v3/users/63353800165546f4971e83489a2a9cdc` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.453041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=63353800165546f4971e83489a2a9cdc)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.455122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.464761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1120/2241] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:24 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:24.466330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.468434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.482213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7f1e5-149b-471b-9d39-417927bab5b4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.487002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.487611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.487887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.488178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.488895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.490498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.500494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1121/2242] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:24.504635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.514399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e3098-3842-414e-bd8e-8091b1f47c5e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.519264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.520234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.520499 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.522095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.523317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.524961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.537258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1122/2243] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:24.541797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.557421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f368f-787f-4fcd-8f73-d4c0623287ec admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.559472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337df779-f223-4803-ba86-fee5f3d109fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.560281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337df779-f223-4803-ba86-fee5f3d109fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.560556 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337df779-f223-4803-ba86-fee5f3d109fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.560821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337df779-f223-4803-ba86-fee5f3d109fb admin admin] PATH_INFO: `/v3/projects/7ca265a4581749b8bad7019efaf24b76/users/074bb39ece3241959db11659ab47f089/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.581310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337df779-f223-4803-ba86-fee5f3d109fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca265a4581749b8bad7019efaf24b76, user_id=074bb39ece3241959db11659ab47f089, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.584112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337df779-f223-4803-ba86-fee5f3d109fb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:24.653664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1123/2244] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:24 2022] PUT /identity/v3/projects/7ca265a4581749b8bad7019efaf24b76/users/074bb39ece3241959db11659ab47f089/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:24.660589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63353800165546f4971e83489a2a9cdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.665735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.666028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.668032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.698723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] MFA Rules not processed for user `074bb39ece3241959db11659ab47f089`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:24.701990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.709161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:24.744451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63353800165546f4971e83489a2a9cdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.754070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63353800165546f4971e83489a2a9cdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.754814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63353800165546f4971e83489a2a9cdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.755792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63353800165546f4971e83489a2a9cdc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:24.788729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1122/2245] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:24 2022] DELETE /identity/v3/users/63353800165546f4971e83489a2a9cdc => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:24.788941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1124/2246] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:24.792286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:24.793687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:24.893377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93a4bb-e579-4abd-b391-bef6194565ad admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:24.903111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:24.903111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:24.903111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:24.903111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:24.903574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:24.905046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.034690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:25.043190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1123/2247] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:24 2022] POST /identity/v3/projects => generated 387 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:25.047364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.149416 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66047a4d-cc7d-47dc-b0a1-612b29df1067 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.220501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.221113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.221393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.221708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.222340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.224351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.275823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.278649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:25.294243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a3d4-a5a6-49c0-b5f0-16a0f5da343c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.302088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.302088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.302811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1125/2248] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:25 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:25.302981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.303178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] PATH_INFO: `/v3/projects/8816be837ab444d0aeaa5bb8a0695e89` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.305244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.313210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8816be837ab444d0aeaa5bb8a0695e89)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.315435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.315547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddb7cc-0571-4f2a-ae58-79dc07cfac3e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.321944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.322739 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.323162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.323596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.324596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.331422 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.342256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1126/2249] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:25.346245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.357176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865cb2e-048d-428a-87d4-6ce7c8964f6c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.361843 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.362527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.362804 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.363073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] PATH_INFO: `/v3/projects/41bcd6fc315f4740a16f76f1699512a4/users/b0a556058de9425daa13f35d7ab6b80e/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.378702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41bcd6fc315f4740a16f76f1699512a4, user_id=b0a556058de9425daa13f35d7ab6b80e, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.381463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.453932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1127/2250] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:25 2022] PUT /identity/v3/projects/41bcd6fc315f4740a16f76f1699512a4/users/b0a556058de9425daa13f35d7ab6b80e/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:25.457983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.461728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8816be837ab444d0aeaa5bb8a0695e89', 'request_id': 'req-3d7343da-dc64-429e-88cc-8c4454e3ce26'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:25.519453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fdb55-d564-44f5-8ae6-6d695df898c9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.521316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8816be837ab444d0aeaa5bb8a0695e89', 'request_id': 'req-3d7343da-dc64-429e-88cc-8c4454e3ce26'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:25.527928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8816be837ab444d0aeaa5bb8a0695e89', 'request_id': 'req-3d7343da-dc64-429e-88cc-8c4454e3ce26'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:25.544619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] The token cache is being invalidate because project 8816be837ab444d0aeaa5bb8a0695e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:25.544857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:25.545519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3d7343da-dc64-429e-88cc-8c4454e3ce26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:25.547198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1124/2251] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:25 2022] DELETE /identity/v3/projects/8816be837ab444d0aeaa5bb8a0695e89 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:25.551888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d815b-df72-4162-9d80-b30afdf7b7c8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.552324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d815b-df72-4162-9d80-b30afdf7b7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.555347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d815b-df72-4162-9d80-b30afdf7b7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.573254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f33d815b-df72-4162-9d80-b30afdf7b7c8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:25.574533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f33d815b-df72-4162-9d80-b30afdf7b7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:25.584123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.585346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.585346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.585572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.586226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.587855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.605968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1128/2252] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:25.609871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.612029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:25.625447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:25.629719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438ed17-f1a8-4614-a90f-9b34c1155693 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.635393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae29e2-dff4-4b30-bcdc-d94a18362d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.636492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae29e2-dff4-4b30-bcdc-d94a18362d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.636849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae29e2-dff4-4b30-bcdc-d94a18362d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.637447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae29e2-dff4-4b30-bcdc-d94a18362d70 admin admin] PATH_INFO: `/v3/projects/41bcd6fc315f4740a16f76f1699512a4/users/b0a556058de9425daa13f35d7ab6b80e/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.639226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae29e2-dff4-4b30-bcdc-d94a18362d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41bcd6fc315f4740a16f76f1699512a4, user_id=b0a556058de9425daa13f35d7ab6b80e, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.642350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae29e2-dff4-4b30-bcdc-d94a18362d70 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.647870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1125/2253] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:25.655788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.656081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.656396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.666881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1129/2254] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:25 2022] PUT /identity/v3/projects/41bcd6fc315f4740a16f76f1699512a4/users/b0a556058de9425daa13f35d7ab6b80e/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:25.677042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:25.679741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:25.680996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.681491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.681926 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.712330 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] MFA Rules not processed for user `b0a556058de9425daa13f35d7ab6b80e`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:25.713821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:25.747726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1126/2255] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:25.757111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.767086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb190f-06ec-4c38-84ba-008b0f601fb8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.772361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.772979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.773257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.773531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.774204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:25.774497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.776186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.808681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1127/2256] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:47:25.808681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.820944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb626d72-5ede-4e80-b6ce-a3747c4ed588 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.826429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.828641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.829101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.836093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.836093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.836413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1130/2257] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:25.836531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.871173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.872713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:25.937975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d89877-d94b-4d32-b4ff-d3c05ef17b26 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:25.941884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:25.942789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:25.946611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:25.946611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:25.953946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1128/2258] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:25 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:25.954152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:25.954863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:25.955487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:25.984245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1131/2259] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:47:26.033466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81670a20-6b6f-4e09-9a4d-b58b5a0c4225 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.039186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.039823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.040048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.040345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.040970 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.042991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.054540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:26.065351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a10eb-bdc3-485f-bc9d-32502f827772 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.071064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.071978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.072300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.072714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] PATH_INFO: `/v3/users/074bb39ece3241959db11659ab47f089` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.074111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074bb39ece3241959db11659ab47f089)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.076451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.137489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1129/2260] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:25 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:26.141333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:26.152329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde03e94-91d7-4030-9e0e-b88e63877cdf admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.157310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.157922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.158337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.158610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.159267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.160952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.171854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1130/2261] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:26.179965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:26.189613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d884c37-08a2-4a73-a3ea-4a5976c68335 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.200577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.201332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.201599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.201874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.202524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.204248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.217108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1131/2262] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:26.221272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:26.231626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aada8-529c-4284-8ee0-7521a48ba37d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.236251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3196677-164f-4560-a7c3-6bc9d785ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.237058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3196677-164f-4560-a7c3-6bc9d785ed7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.237336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3196677-164f-4560-a7c3-6bc9d785ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.237607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3196677-164f-4560-a7c3-6bc9d785ed7b admin admin] PATH_INFO: `/v3/projects/2497a9df3f354a5789cc9f7735e39828/users/0fb67f4c58934a568fedf74dc9d45c8d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.252382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3196677-164f-4560-a7c3-6bc9d785ed7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2497a9df3f354a5789cc9f7735e39828, user_id=0fb67f4c58934a568fedf74dc9d45c8d, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.255637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3196677-164f-4560-a7c3-6bc9d785ed7b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.287557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074bb39ece3241959db11659ab47f089'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:26.288863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074bb39ece3241959db11659ab47f089'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:26.289451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074bb39ece3241959db11659ab47f089'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:26.356027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1132/2263] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:26 2022] PUT /identity/v3/projects/2497a9df3f354a5789cc9f7735e39828/users/0fb67f4c58934a568fedf74dc9d45c8d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:26.361022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.361305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.361577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.378947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074bb39ece3241959db11659ab47f089'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:26.382198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:26.382198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:26.382525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:26.394210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f956e376-9939-470f-a493-917dc419ccc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074bb39ece3241959db11659ab47f089'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:26.414094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1132/2264] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:26 2022] DELETE /identity/v3/users/074bb39ece3241959db11659ab47f089 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:26.418527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.418809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.419080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.437599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:26.438797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:26.448890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1133/2265] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:26.460799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:26.462190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:26.499179 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1133/2266] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:26.514094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:26.521778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fda7a7-e00b-4cf3-98f5-5db6b2e30f46 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.555246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:26.574336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.575264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.575750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.575873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d9ecb-d2e0-4cf3-b30c-afa52a8c77cf None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:26.576242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.582740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:26.582740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:26.582740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:26.582740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:26.589211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.590752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.594395 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:26.595830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:26.601474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1134/2267] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:26.606903 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1134/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:47:27.096608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:27.098922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:27.125934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87b241-1f73-4d1b-9438-e3fd1f4d854b service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:27.136017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:27.137422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:27.137934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:27.138426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] PATH_INFO: `/v3/projects/7ca265a4581749b8bad7019efaf24b76` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:27.140383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca265a4581749b8bad7019efaf24b76)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:27.145082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:27.184585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:27.274713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca265a4581749b8bad7019efaf24b76', 'request_id': 'req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.333429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:27.380840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca265a4581749b8bad7019efaf24b76', 'request_id': 'req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.389173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca265a4581749b8bad7019efaf24b76', 'request_id': 'req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.405465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] The token cache is being invalidate because project 7ca265a4581749b8bad7019efaf24b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:27.405850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.406840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-12fd9c94-1a79-4661-95c9-74ceb02bf8ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.410529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1135/2269] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:27 2022] DELETE /identity/v3/projects/7ca265a4581749b8bad7019efaf24b76 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:27.415747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:27.426702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55b644-2525-412e-8d3f-e7e57fb79439 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:27.487914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:27.488587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:27.488869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:27.489145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] PATH_INFO: `/v3/users/b0a556058de9425daa13f35d7ab6b80e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:27.490271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a556058de9425daa13f35d7ab6b80e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:27.492291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:27.700308 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a556058de9425daa13f35d7ab6b80e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.700996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a556058de9425daa13f35d7ab6b80e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.701574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a556058de9425daa13f35d7ab6b80e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.853537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a556058de9425daa13f35d7ab6b80e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.863115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a556058de9425daa13f35d7ab6b80e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:27.882708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1135/2270] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:27 2022] DELETE /identity/v3/users/b0a556058de9425daa13f35d7ab6b80e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:47:28.248370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.248929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.248929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.280775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] MFA Rules not processed for user `0fb67f4c58934a568fedf74dc9d45c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:28.282002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:28.328781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:28.330217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:28.359198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1136/2271] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:28.371091 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:28.372859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:28.385114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:28.415363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce074619-1045-465b-a199-b06d6a00db70 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:28.421676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:28.422358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.422644 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.422918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] PATH_INFO: `/v3/projects/41bcd6fc315f4740a16f76f1699512a4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.424083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=41bcd6fc315f4740a16f76f1699512a4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:28.426104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:28.435726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:28.452518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde3892-379e-408b-a170-ba7f2a501b54 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:28.458634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd747f-e5ee-422a-9cb5-67e40c1fc45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:28.459244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd747f-e5ee-422a-9cb5-67e40c1fc45b service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.459518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd747f-e5ee-422a-9cb5-67e40c1fc45b service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.460342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd747f-e5ee-422a-9cb5-67e40c1fc45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.470126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd747f-e5ee-422a-9cb5-67e40c1fc45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:28.471621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd747f-e5ee-422a-9cb5-67e40c1fc45b service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:28.481245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1137/2272] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:47:28.561398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41bcd6fc315f4740a16f76f1699512a4', 'request_id': 'req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:28.642009 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41bcd6fc315f4740a16f76f1699512a4', 'request_id': 'req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:28.651099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41bcd6fc315f4740a16f76f1699512a4', 'request_id': 'req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:28.671352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] The token cache is being invalidate because project 41bcd6fc315f4740a16f76f1699512a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:28.672041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:28.673278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-06ec93b9-5c43-44ee-b9c6-ba36268f7a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:28.675359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1136/2273] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:28 2022] DELETE /identity/v3/projects/41bcd6fc315f4740a16f76f1699512a4 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:28.684355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.684355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.684355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.707332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:28.708814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:28.787312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1138/2274] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:28.795601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.796098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.796224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.817995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:28.819891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:28.830058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1137/2275] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:28.834624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:28.849747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec966c-486c-4ab2-a21d-86a6edbc1c82 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:28.855058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:28.855885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.856079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.856392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.857137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:28.857436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:28.859153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:28.874214 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1139/2276] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:47:28.878432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:28.890461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5382f2a-303d-4be1-98bf-9e0b5e813438 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:28.897662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:28.898274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:28.900068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:28.900068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:28.900068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:28.901977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:28.988023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1138/2277] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:28 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:28.992781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.003559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf4e0c-5b68-4058-a792-b6fa0606302d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.074723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.075334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.075614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.075884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.076562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.078569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.159052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1140/2278] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:28 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:29.163662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.176533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f743a6-7ae7-4e56-a974-3a3974f39222 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.182129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.182858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.183198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.183534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.184325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.186086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.196910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1139/2279] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:29.202860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.215616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb516-ea21-4d3e-a4e2-beb852cf108e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.220946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130db7ed-4f6d-4b0a-9ab4-4d0e4d6d2044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.221707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130db7ed-4f6d-4b0a-9ab4-4d0e4d6d2044 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.221985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130db7ed-4f6d-4b0a-9ab4-4d0e4d6d2044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.222285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130db7ed-4f6d-4b0a-9ab4-4d0e4d6d2044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.224502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130db7ed-4f6d-4b0a-9ab4-4d0e4d6d2044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.224758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130db7ed-4f6d-4b0a-9ab4-4d0e4d6d2044 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.240485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1141/2280] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:29.251688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.256751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf2fc1-ebe3-4060-a533-50d2bc025edd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.262562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.263249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.263529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.263820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] PATH_INFO: `/v3/projects/39472977e6d84a289bdf5b4ad40fab46/users/6c98d7c186bb4a84bab9d5495e830dbe/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.282525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=39472977e6d84a289bdf5b4ad40fab46, user_id=6c98d7c186bb4a84bab9d5495e830dbe, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.285561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.362377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1140/2281] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:29 2022] PUT /identity/v3/projects/39472977e6d84a289bdf5b4ad40fab46/users/6c98d7c186bb4a84bab9d5495e830dbe/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:29.373651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.374143 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.374254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.396461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:29.403732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] MFA Rules not processed for user `6c98d7c186bb4a84bab9d5495e830dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:29.405046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:29.469797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1142/2282] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:29.474458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.543338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9405bc6-c76f-4a09-836d-3494cdd572bb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.548060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.548699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.548976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.549249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.549909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.551821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.667590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1141/2283] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:29 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:29.670238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.681817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf9da9-0f97-4ecd-9405-456a2a7281e2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.745575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.746199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.746481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.746767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.747413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.749422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.848588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1143/2284] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:29 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:29.852997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.867382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3f097-c700-41dd-9485-07bee03fe88f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.873175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.873907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.874186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.874453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.875405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.877460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:29.891837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1142/2285] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:29.896238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:29.907255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e46779-39b4-40da-be57-1e9e7cc1efd1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:29.912286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:29.912975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:29.913253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:29.913526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] PATH_INFO: `/v3/projects/2449684009134d84bad572d34b58069a/users/4b48947af25a4012843f873b1dbadc75/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:29.932436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2449684009134d84bad572d34b58069a, user_id=4b48947af25a4012843f873b1dbadc75, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:29.936450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:30.058719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1144/2286] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:29 2022] PUT /identity/v3/projects/2449684009134d84bad572d34b58069a/users/4b48947af25a4012843f873b1dbadc75/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:30.062879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:30.070322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:30.074156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc5b13-f622-4b3b-844d-b939b54d6077 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:30.132962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d1fa9-2094-4add-afb9-bea0bf0e17b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:30.133533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d1fa9-2094-4add-afb9-bea0bf0e17b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:30.133814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d1fa9-2094-4add-afb9-bea0bf0e17b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:30.134084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d1fa9-2094-4add-afb9-bea0bf0e17b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:30.134788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d1fa9-2094-4add-afb9-bea0bf0e17b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:30.140826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d1fa9-2094-4add-afb9-bea0bf0e17b0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:30.146912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1143/2287] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:30.151686 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:30.162264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c380f6-0283-4a86-8d85-22fe72cd2597 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:30.167823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:30.170651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:30.170651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:30.170651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] PATH_INFO: `/v3/projects/2449684009134d84bad572d34b58069a/users/4b48947af25a4012843f873b1dbadc75/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:30.171372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2449684009134d84bad572d34b58069a, user_id=4b48947af25a4012843f873b1dbadc75, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:30.174108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:30.185393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1145/2288] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:30 2022] PUT /identity/v3/projects/2449684009134d84bad572d34b58069a/users/4b48947af25a4012843f873b1dbadc75/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:30.196822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395a875-563a-4c2c-a177-b1eb0358d0a0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:30.197110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395a875-563a-4c2c-a177-b1eb0358d0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:30.199564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395a875-563a-4c2c-a177-b1eb0358d0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:30.228446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c395a875-563a-4c2c-a177-b1eb0358d0a0 None None] MFA Rules not processed for user `4b48947af25a4012843f873b1dbadc75`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:30.229682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c395a875-563a-4c2c-a177-b1eb0358d0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:30.308274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1144/2289] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:30.430049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:30.431832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:30.434859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:30.434859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:30.439600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:30.456784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] MFA Rules not processed for user `0fb67f4c58934a568fedf74dc9d45c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:30.458089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:30.508496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229abc2-16ef-458f-9902-7021d747c9c2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:30.512858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:30.513458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:30.513735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:30.514002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:30.523168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:30.524117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1145/2290] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:30.526085 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:30.527709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:30.544709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1146/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:47:30.758123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:30.767923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42b32a-3f73-4036-aa5f-e6e7572751c3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:30.771601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3de29e-106b-42db-826f-12258b8de002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:30.772729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3de29e-106b-42db-826f-12258b8de002 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:30.773014 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3de29e-106b-42db-826f-12258b8de002 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:30.773285 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3de29e-106b-42db-826f-12258b8de002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:30.804388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:30.835232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3de29e-106b-42db-826f-12258b8de002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:30.836718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3de29e-106b-42db-826f-12258b8de002 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:30.845564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1146/2292] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:39.909532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:40.009206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d97d-64d7-4242-86ba-f86ba802cac7 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:40.014906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:40.015795 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:40.016496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:40.017039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] PATH_INFO: `/v3/users/6c98d7c186bb4a84bab9d5495e830dbe` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:40.018200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c98d7c186bb4a84bab9d5495e830dbe)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:40.020137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:40.216263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c98d7c186bb4a84bab9d5495e830dbe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:40.308630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c98d7c186bb4a84bab9d5495e830dbe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:40.317170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c98d7c186bb4a84bab9d5495e830dbe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:40.318075 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c98d7c186bb4a84bab9d5495e830dbe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:40.318808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c98d7c186bb4a84bab9d5495e830dbe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:40.344708 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1147/2293] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:39 2022] DELETE /identity/v3/users/6c98d7c186bb4a84bab9d5495e830dbe => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:40.346812 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76bd5b-5cdb-48d5-affe-933cab267c7c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:40.351224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76bd5b-5cdb-48d5-affe-933cab267c7c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:40.351224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76bd5b-5cdb-48d5-affe-933cab267c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:40.369331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8c76bd5b-5cdb-48d5-affe-933cab267c7c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:40.371331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8c76bd5b-5cdb-48d5-affe-933cab267c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:40.460323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1147/2294] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:40.470534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:40.472073 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:40.555972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6162b0-bf1a-4ad2-b9d5-ba299654f28f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:40.562444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:40.563144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:40.563412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:40.563837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:40.575639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:40.577218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:40.589527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1148/2295] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:40.738720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:40.739001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:40.739279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:40.760011 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] MFA Rules not processed for user `8eb5509df78d45acbebc332a889251dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:40.761304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:40.825110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1148/2296] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:41.066241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:41.067936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:41.088238 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1ac4-121b-40c1-8d42-0ddd758accef service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:41.093230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:41.093879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:41.094158 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:41.094429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] PATH_INFO: `/v3/projects/39472977e6d84a289bdf5b4ad40fab46` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:41.095687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=39472977e6d84a289bdf5b4ad40fab46)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:41.097728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:41.111303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:41.196508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39472977e6d84a289bdf5b4ad40fab46', 'request_id': 'req-60430aed-a639-49df-9959-3f066add4aef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.304721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39472977e6d84a289bdf5b4ad40fab46', 'request_id': 'req-60430aed-a639-49df-9959-3f066add4aef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.311199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39472977e6d84a289bdf5b4ad40fab46', 'request_id': 'req-60430aed-a639-49df-9959-3f066add4aef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.326081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] The token cache is being invalidate because project 39472977e6d84a289bdf5b4ad40fab46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:41.326493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.327213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.328885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1149/2297] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:41 2022] DELETE /identity/v3/projects/39472977e6d84a289bdf5b4ad40fab46 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:41.333077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:41.344125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48032122-93b6-431a-9b16-5b4e35bc3003 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:41.409024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:41.409771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:41.410178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:41.410686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] PATH_INFO: `/v3/users/4b48947af25a4012843f873b1dbadc75` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:41.411929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b48947af25a4012843f873b1dbadc75)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:41.414674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:41.432814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:41.576301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b48947af25a4012843f873b1dbadc75'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.577237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b48947af25a4012843f873b1dbadc75'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.577848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b48947af25a4012843f873b1dbadc75'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.656937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b48947af25a4012843f873b1dbadc75'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.665720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6e884f-a098-41e9-ac4f-335d1ebf8447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b48947af25a4012843f873b1dbadc75'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:41.684091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1149/2298] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:41 2022] DELETE /identity/v3/users/4b48947af25a4012843f873b1dbadc75 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:42.185640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:42.187654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:42.202575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60430aed-a639-49df-9959-3f066add4aef admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:42.266206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:42.267208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:42.267345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:42.267611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] PATH_INFO: `/v3/projects/2449684009134d84bad572d34b58069a` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:42.268796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2449684009134d84bad572d34b58069a)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:42.270717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:42.458114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2449684009134d84bad572d34b58069a', 'request_id': 'req-441d1932-69ad-428a-8b57-30a27840c1a6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:42.521282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2449684009134d84bad572d34b58069a', 'request_id': 'req-441d1932-69ad-428a-8b57-30a27840c1a6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:42.527862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2449684009134d84bad572d34b58069a', 'request_id': 'req-441d1932-69ad-428a-8b57-30a27840c1a6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:42.539529 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] The token cache is being invalidate because project 2449684009134d84bad572d34b58069a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:42.539892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:42.540752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-441d1932-69ad-428a-8b57-30a27840c1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:42.542515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1150/2299] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:42 2022] DELETE /identity/v3/projects/2449684009134d84bad572d34b58069a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:42.547152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:42.547625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:42.547718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:42.571148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:42.572282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:42.635254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1150/2300] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:47:42.643368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:42.643687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:42.643986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:42.663514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:42.664750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:42.672568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1151/2301] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:42.676784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:42.687057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9b588-21c8-44b2-8656-a8ae92479521 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:42.691788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:42.692459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:42.692854 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:42.693136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:42.694428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:47:42.694737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:42.696365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:42.707974 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1151/2302] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:47:42.712203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:42.721639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b828551-abdc-4e38-8f03-550e967a873c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:42.726506 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:42.727131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:42.727428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:42.727728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:42.728554 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:42.730469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:42.826701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1152/2303] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:42 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:42.831898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:42.843947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411f4db-7a13-4d8f-8f80-3e9acab8e5b8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:42.911477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:42.912706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:42.912899 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:42.913174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:42.913864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:42.915777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:43.043314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1152/2304] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:42 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:47:43.045559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:43.053688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:43.065859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbb881-bff3-433c-a244-051b3a5a4c35 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:43.095145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:43.095145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:43.095145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:43.095145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:43.095480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:43.095480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:43.106064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1153/2305] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:43.110154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:43.136825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62877d1-14d9-4df8-b846-f315af734dd0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:43.136825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7882c-07a6-4eb2-a7fc-176ebe188080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:43.136825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7882c-07a6-4eb2-a7fc-176ebe188080 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:43.136825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7882c-07a6-4eb2-a7fc-176ebe188080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:43.137173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7882c-07a6-4eb2-a7fc-176ebe188080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:43.137173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7882c-07a6-4eb2-a7fc-176ebe188080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:43.138613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7882c-07a6-4eb2-a7fc-176ebe188080 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:43.145841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:43.159372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1153/2306] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:47:43.160245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:43.181928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ce7c1-fc7f-4ab0-b974-bb5d606e715e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:43.202849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:43.204208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:43.204641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:43.205103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] PATH_INFO: `/v3/projects/db1b940b522145b98ec2ef13d2639bb4/users/48066448361e4004ac6ef4722fc91141/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:43.227479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1b940b522145b98ec2ef13d2639bb4, user_id=48066448361e4004ac6ef4722fc91141, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:43.231156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:43.306289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1154/2307] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:43 2022] PUT /identity/v3/projects/db1b940b522145b98ec2ef13d2639bb4/users/48066448361e4004ac6ef4722fc91141/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:43.311630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e411b-8e1f-4aaa-94ab-0b713dbeddaa None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:43.311987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e411b-8e1f-4aaa-94ab-0b713dbeddaa None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:43.312415 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e411b-8e1f-4aaa-94ab-0b713dbeddaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:43.332781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d21e411b-8e1f-4aaa-94ab-0b713dbeddaa None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:43.333990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d21e411b-8e1f-4aaa-94ab-0b713dbeddaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:43.413287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1154/2308] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:43.422880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:43.424756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:43.486636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36b65-d35d-4369-a565-fb55aa19c04e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:43.491257 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:43.491870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:43.492182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:43.492450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:43.502655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:43.504250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:43.518984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1155/2309] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:45.031186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:45.031709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:45.031823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:45.067954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] MFA Rules not processed for user `48066448361e4004ac6ef4722fc91141`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:45.070288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:45.177453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1155/2310] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:45.177548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:45.191496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cfdeb-1ebb-4757-8e08-b7d2c6afbc15 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:45.198574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:45.204290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:45.204290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:45.204290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:45.204290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:45.204290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:45.315203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1156/2311] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:47:45 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:45.330971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:45.333470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e335a-486e-435d-bed2-eb7b802cf8ff None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:45.463518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:45.464425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:45.464785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:45.465063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:45.466426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:45.469266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:45.572227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1156/2312] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:47:45 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:47:45.574934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:45.588197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bffe63-beba-4343-9984-3748efb529a9 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:45.595319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:45.597584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:45.598015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:45.598465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:45.599263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:45.601343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:45.629019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1157/2313] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:45.633771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:45.645913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb95838-d846-484f-bf9d-1b3c91168ce6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:45.651618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:45.677068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:45.677068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:45.677068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] PATH_INFO: `/v3/projects/b404542da5144e78a8ac49e3e1d01604/users/a1cd571ca60f4cd8a5b29786ddf6bd4c/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:45.704047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b404542da5144e78a8ac49e3e1d01604, user_id=a1cd571ca60f4cd8a5b29786ddf6bd4c, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:45.707181 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:45.711696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:45.864194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:45.865296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1157/2314] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:45 2022] PUT /identity/v3/projects/b404542da5144e78a8ac49e3e1d01604/users/a1cd571ca60f4cd8a5b29786ddf6bd4c/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:45.876376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5d3cf-0ec0-459a-bf54-2480e05705a5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:45.945123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460ac0-2392-4e6d-84d0-2e9a3a1add84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:45.951135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460ac0-2392-4e6d-84d0-2e9a3a1add84 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:45.951135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460ac0-2392-4e6d-84d0-2e9a3a1add84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:45.951135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460ac0-2392-4e6d-84d0-2e9a3a1add84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:45.951135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac460ac0-2392-4e6d-84d0-2e9a3a1add84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:45.952658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac460ac0-2392-4e6d-84d0-2e9a3a1add84 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:45.976871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1158/2315] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:47:45.981526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:45.996124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac24f3-68e0-477c-a4bb-3fa082fd902d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:46.022474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:46.023227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:46.023538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:46.027019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] PATH_INFO: `/v3/projects/b404542da5144e78a8ac49e3e1d01604/users/a1cd571ca60f4cd8a5b29786ddf6bd4c/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:46.033104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b404542da5144e78a8ac49e3e1d01604, user_id=a1cd571ca60f4cd8a5b29786ddf6bd4c, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:46.035951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:46.050685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1158/2316] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:47:45 2022] PUT /identity/v3/projects/b404542da5144e78a8ac49e3e1d01604/users/a1cd571ca60f4cd8a5b29786ddf6bd4c/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:47.787169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:47.787942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:47.787942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:47.831744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] MFA Rules not processed for user `a1cd571ca60f4cd8a5b29786ddf6bd4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:47.833782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:47.860637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:47.932344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1159/2317] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:47.949504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:47.951551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:48.019067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69ecab-8b2b-4d9c-888e-fdff12299051 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:48.022995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:48.023604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:48.023881 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:48.024172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:48.080891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:48.082457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:48.092101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1159/2318] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:50.219671 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:50.221157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:50.237625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4959b68-7678-45f1-8ab1-df02e6d74cfb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:50.237625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b51937-006f-4003-b318-b85a56d26171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:50.238234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51937-006f-4003-b318-b85a56d26171 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:50.238519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51937-006f-4003-b318-b85a56d26171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:50.238794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51937-006f-4003-b318-b85a56d26171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:50.253552 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b51937-006f-4003-b318-b85a56d26171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:50.255140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b51937-006f-4003-b318-b85a56d26171 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:50.267649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1160/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:55.227752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:55.229231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:55.315709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f741aa9-0978-49ba-8129-dbf2eab839fd service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:55.320596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:55.321931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:55.321931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:55.321931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] PATH_INFO: `/v3/users/9b044e9de07e4a1c80f8fbb94c66ca55` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:55.323084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b044e9de07e4a1c80f8fbb94c66ca55)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:55.325098 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:55.520777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:47:55.534743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b044e9de07e4a1c80f8fbb94c66ca55'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:55.536588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b044e9de07e4a1c80f8fbb94c66ca55'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:55.537628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b044e9de07e4a1c80f8fbb94c66ca55'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:55.606831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b044e9de07e4a1c80f8fbb94c66ca55'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:55.619199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b044e9de07e4a1c80f8fbb94c66ca55'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:55.642600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1160/2320] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:47:55 2022] DELETE /identity/v3/users/9b044e9de07e4a1c80f8fbb94c66ca55 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:47:55.647146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:55.647660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:55.647660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:55.672823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:47:55.672823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:55.738984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1161/2321] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:47:55.748993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:55.750413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:55.809990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a037-0166-45f6-bb39-01ffbeec80b1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:55.813614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:55.814227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:55.814509 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:55.814784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:55.825900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:55.828138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:55.838545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1161/2322] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:47:56.413181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:47:56.427726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:47:56.447119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c822cfb-722c-4a2a-a2af-007be4701857 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:47:56.452280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:47:56.452912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:47:56.453191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:47:56.453463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] PATH_INFO: `/v3/projects/d92aee0ad4664de698592bf4cc20b5d6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:47:56.454691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92aee0ad4664de698592bf4cc20b5d6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:47:56.456663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:47:56.590076 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92aee0ad4664de698592bf4cc20b5d6', 'request_id': 'req-e98273fe-b87b-42ce-831e-f763a538bd7a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:56.658193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92aee0ad4664de698592bf4cc20b5d6', 'request_id': 'req-e98273fe-b87b-42ce-831e-f763a538bd7a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:56.667682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92aee0ad4664de698592bf4cc20b5d6', 'request_id': 'req-e98273fe-b87b-42ce-831e-f763a538bd7a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:56.683079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] The token cache is being invalidate because project d92aee0ad4664de698592bf4cc20b5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:47:56.683465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:56.684270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:47:56.694403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1162/2323] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:47:56 2022] DELETE /identity/v3/projects/d92aee0ad4664de698592bf4cc20b5d6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:06.578112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:06.679774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:07.547574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:07.549246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:07.562898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfecab-8384-4f8e-80ca-fdc4b47e480b service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:07.626375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:07.627137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:07.627521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:07.627897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] PATH_INFO: `/v3/users/48066448361e4004ac6ef4722fc91141` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:07.629447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48066448361e4004ac6ef4722fc91141)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:07.631910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:07.887731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48066448361e4004ac6ef4722fc91141'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:07.888513 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48066448361e4004ac6ef4722fc91141'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:07.889218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48066448361e4004ac6ef4722fc91141'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:07.961534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48066448361e4004ac6ef4722fc91141'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:07.974635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48066448361e4004ac6ef4722fc91141'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:07.994602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1162/2324] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:48:07 2022] DELETE /identity/v3/users/48066448361e4004ac6ef4722fc91141 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:08.243051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:08.245137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:08.260498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98273fe-b87b-42ce-831e-f763a538bd7a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:08.325006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:08.325935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:08.326371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:08.326950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] PATH_INFO: `/v3/projects/db1b940b522145b98ec2ef13d2639bb4` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:08.328299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1b940b522145b98ec2ef13d2639bb4)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:08.330533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:08.463824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1b940b522145b98ec2ef13d2639bb4', 'request_id': 'req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:08.548546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1b940b522145b98ec2ef13d2639bb4', 'request_id': 'req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:08.559675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1b940b522145b98ec2ef13d2639bb4', 'request_id': 'req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:08.573421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] The token cache is being invalidate because project db1b940b522145b98ec2ef13d2639bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:48:08.573805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:08.574685 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:08.577190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1163/2325] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:48:08 2022] DELETE /identity/v3/projects/db1b940b522145b98ec2ef13d2639bb4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:08.580605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:08.587235 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:08.606529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6331a3-3e91-470c-8ab2-e6fbc55ff677 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:08.701559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:08.702959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:08.703243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:08.703520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] PATH_INFO: `/v3/users/a1cd571ca60f4cd8a5b29786ddf6bd4c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:08.704813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1cd571ca60f4cd8a5b29786ddf6bd4c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:08.706946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:08.734468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:09.030944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1cd571ca60f4cd8a5b29786ddf6bd4c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.031647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1cd571ca60f4cd8a5b29786ddf6bd4c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.032267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1cd571ca60f4cd8a5b29786ddf6bd4c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.130658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1cd571ca60f4cd8a5b29786ddf6bd4c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.138392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e019b09-2061-4a2d-b9af-fac97ae836de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1cd571ca60f4cd8a5b29786ddf6bd4c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.157342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1163/2326] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:48:08 2022] DELETE /identity/v3/users/a1cd571ca60f4cd8a5b29786ddf6bd4c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:09.458083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:09.459528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:09.474556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b368d4-793e-4d1e-aaa6-e4315e59b29c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:09.540802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:09.541471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:09.541749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:09.542021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] PATH_INFO: `/v3/projects/b404542da5144e78a8ac49e3e1d01604` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:09.543150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b404542da5144e78a8ac49e3e1d01604)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:09.545272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:09.652635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:09.664217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b404542da5144e78a8ac49e3e1d01604', 'request_id': 'req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.744667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b404542da5144e78a8ac49e3e1d01604', 'request_id': 'req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.750677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b404542da5144e78a8ac49e3e1d01604', 'request_id': 'req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.763182 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] The token cache is being invalidate because project b404542da5144e78a8ac49e3e1d01604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:48:09.763464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.764237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4ead1f91-5b34-40bb-b1a0-47a3296d82a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:09.766386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1164/2327] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:48:09 2022] DELETE /identity/v3/projects/b404542da5144e78a8ac49e3e1d01604 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:09.772195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:09.772708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:09.772902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:09.833979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:09.835564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:09.931986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1164/2328] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:09.934743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:09.935024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:09.935294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:09.963028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:09.965028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:09.979665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1165/2329] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:09.997635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.025275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89d852-6f28-4378-93ad-aae670b552cb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.034941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.035557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.035838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.036105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.036880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:48:10.037184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.039595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.076964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.081726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1165/2330] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:48:10.086987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184364d-509a-46b5-b6f0-8e6227fdd233 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.092258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.092960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.093199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.093469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.094104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.096346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.185017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1166/2331] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:48:10 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:48:10.189351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.201409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b2c9b-8fec-4f39-b193-26454d6ea431 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.262358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.262901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.263182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.263454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.264094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.266087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.356645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1166/2332] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:48:10 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:48:10.360670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.371222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd1771-c38a-4712-b8ec-f7284777adc5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.375947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.376577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.383724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.383724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.383724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.383724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.389602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1167/2333] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:48:10.393912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.404847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938030e0-a6f0-49e4-bf48-340f6bfe0fc3 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.409639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da460c41-7039-4427-bb2f-375d7d08de44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.410234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da460c41-7039-4427-bb2f-375d7d08de44 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.410654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da460c41-7039-4427-bb2f-375d7d08de44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.410865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da460c41-7039-4427-bb2f-375d7d08de44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.411530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da460c41-7039-4427-bb2f-375d7d08de44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.413168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da460c41-7039-4427-bb2f-375d7d08de44 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.426435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1167/2334] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:48:10.430284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.453097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563278-7ad0-451f-902d-89f204d6bb0c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.456372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.457066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.457343 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.457613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] PATH_INFO: `/v3/projects/c7e0ace10fe241aba9f2ab06d05b0032/users/133ae50f4b814e978c875bf61a9db611/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.474542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e0ace10fe241aba9f2ab06d05b0032, user_id=133ae50f4b814e978c875bf61a9db611, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.477990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.527166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1168/2335] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:48:10 2022] PUT /identity/v3/projects/c7e0ace10fe241aba9f2ab06d05b0032/users/133ae50f4b814e978c875bf61a9db611/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:10.538719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.539187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.539512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.547848 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:10.567419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] MFA Rules not processed for user `133ae50f4b814e978c875bf61a9db611`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:10.568717 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:10.634193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1168/2336] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:48:10.650471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:10.652346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:10.714274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cde96-c28f-4f0a-9ffa-07155bf93d9e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:10.718002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0564b-e052-415c-8d61-827cef90e51b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:10.718603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0564b-e052-415c-8d61-827cef90e51b service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:10.718911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0564b-e052-415c-8d61-827cef90e51b service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:10.725934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0564b-e052-415c-8d61-827cef90e51b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:10.727592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0564b-e052-415c-8d61-827cef90e51b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:10.729074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0564b-e052-415c-8d61-827cef90e51b service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:10.737758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1169/2337] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:48:14.207670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:14.209148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:14.281436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a5c82-9bfc-42a2-9258-9ee7543aa2ef None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:14.286025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:14.287173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:14.287462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:14.287736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] PATH_INFO: `/v3/users/17b1ae55272b45488d611968e4cdfc76` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:14.289551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b1ae55272b45488d611968e4cdfc76)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:14.291657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:14.455430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b1ae55272b45488d611968e4cdfc76'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:14.456251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b1ae55272b45488d611968e4cdfc76'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:14.456982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b1ae55272b45488d611968e4cdfc76'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:14.511839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73754296-d6a0-4387-a709-1ad13a6ffc65 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:14.511839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73754296-d6a0-4387-a709-1ad13a6ffc65 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:14.511839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73754296-d6a0-4387-a709-1ad13a6ffc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:14.550188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b1ae55272b45488d611968e4cdfc76'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:14.561290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9501b80e-eea3-49c1-8651-9b5001bc8f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b1ae55272b45488d611968e4cdfc76'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:14.584771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-73754296-d6a0-4387-a709-1ad13a6ffc65 None None] MFA Rules not processed for user `133ae50f4b814e978c875bf61a9db611`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:14.596811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1169/2338] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:48:14 2022] DELETE /identity/v3/users/17b1ae55272b45488d611968e4cdfc76 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:48:14.596811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570f61-9df7-4a65-9aa7-c296b059b266 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:14.596811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570f61-9df7-4a65-9aa7-c296b059b266 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:14.596811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570f61-9df7-4a65-9aa7-c296b059b266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:14.599992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-73754296-d6a0-4387-a709-1ad13a6ffc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:14.628182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-42570f61-9df7-4a65-9aa7-c296b059b266 None None] MFA Rules not processed for user `0fb67f4c58934a568fedf74dc9d45c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:14.630408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-42570f61-9df7-4a65-9aa7-c296b059b266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:14.640473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:14.720696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1170/2339] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:14.726937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:14.730015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:14.730506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:14.746983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1170/2340] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:48:14.779379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:14.786005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:14.796017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:14.796546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:14.796985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:14.807522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:14.831580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:14.833825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:14.868411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1171/2341] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:14.889284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:14.893906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:14.923672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1171/2342] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:48:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:48:14.984673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:14.986326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:15.017189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106bb50-8871-4826-82db-d3c6aa809832 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:15.027226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:15.029902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:15.029902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:15.029902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:15.043453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:15.044951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:15.067062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1172/2343] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:48:15.081193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622dbb8-46b1-4c8f-8e52-66302ab489f9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:15.092510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1be77-ac9a-478d-bb9a-70d016c0d11d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:15.092510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1be77-ac9a-478d-bb9a-70d016c0d11d service swift] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:15.092510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1be77-ac9a-478d-bb9a-70d016c0d11d service swift] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:15.092510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1be77-ac9a-478d-bb9a-70d016c0d11d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:15.104029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1be77-ac9a-478d-bb9a-70d016c0d11d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:15.106395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1be77-ac9a-478d-bb9a-70d016c0d11d service swift] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:15.132604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1172/2344] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:48:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 14:48:16.265001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:16.266460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:16.295859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b50e5d-4bbf-46a2-8050-2690d2c617e4 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:16.308048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:16.310679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:16.310889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:16.311217 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] PATH_INFO: `/v3/projects/b5e50b041ba6416db1001d9b2312f794` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:16.317124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e50b041ba6416db1001d9b2312f794)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:16.319094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:16.508118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e50b041ba6416db1001d9b2312f794', 'request_id': 'req-7d029873-32f4-44a3-8df5-2b0824675627'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:16.590280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e50b041ba6416db1001d9b2312f794', 'request_id': 'req-7d029873-32f4-44a3-8df5-2b0824675627'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:16.591951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:16.598447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e50b041ba6416db1001d9b2312f794', 'request_id': 'req-7d029873-32f4-44a3-8df5-2b0824675627'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:16.618583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] The token cache is being invalidate because project b5e50b041ba6416db1001d9b2312f794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:48:16.618880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:16.624665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7d029873-32f4-44a3-8df5-2b0824675627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:16.627119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1173/2345] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:48:16 2022] DELETE /identity/v3/projects/b5e50b041ba6416db1001d9b2312f794 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:16.633770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:16.644065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:16.644065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:16.662388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:16.666993 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:16.751284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1173/2346] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:16.767872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:16.767872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:16.767872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:16.788390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:16.790617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:16.801102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1174/2347] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:16.806346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:16.817038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facbe87-7b1c-48c6-9f37-af005b2439ca None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:16.822820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:16.823451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:16.823725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:16.823994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:16.824674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:48:16.824974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:16.826577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:16.842187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1174/2348] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:48:16.846626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:16.859232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a978a4a-a8dc-4e06-a30c-4280efaec932 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:16.870599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:16.882836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:16.882836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:16.882836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:16.882836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:16.882836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:17.024184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1175/2349] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:48:16 2022] POST /identity/v3/projects => generated 391 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:48:17.028596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:17.041052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9be46-5dc6-46d9-b80c-84d5006105f7 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:17.069034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:17.127767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:17.128580 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:17.129172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:17.129567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:17.130306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:17.132932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:17.243295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1175/2350] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:48:17 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:48:17.248963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:17.262779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be69012-3db4-493a-bec1-32c2473ece9b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:17.269174 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:17.269914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:17.270194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:17.270890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:17.271774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:17.274643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:17.289237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1176/2351] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:48:17.293666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:17.307988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620edfba-4445-4f18-a9e3-886df13cfed4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:17.313365 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff4e8f-9cc8-44bf-8e86-ddfff63db8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:17.313962 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff4e8f-9cc8-44bf-8e86-ddfff63db8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:17.314236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff4e8f-9cc8-44bf-8e86-ddfff63db8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:17.314501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff4e8f-9cc8-44bf-8e86-ddfff63db8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:17.316191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff4e8f-9cc8-44bf-8e86-ddfff63db8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:17.316799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff4e8f-9cc8-44bf-8e86-ddfff63db8ff admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:17.328692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1176/2352] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:48:17.332598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:17.347414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded5243-3766-4e60-afe7-a5ddee4cd416 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:17.353310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:17.358968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:17.358968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:17.358968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] PATH_INFO: `/v3/projects/104984d8dc7b4055b36b8e6151f2f45f/users/dd767194597c4aa6ae36fa2c21330aba/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:17.374580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] RBAC: Authorizing `identity:create_grant(project_id=104984d8dc7b4055b36b8e6151f2f45f, user_id=dd767194597c4aa6ae36fa2c21330aba, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:17.378487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:17.484091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1177/2353] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:48:17 2022] PUT /identity/v3/projects/104984d8dc7b4055b36b8e6151f2f45f/users/dd767194597c4aa6ae36fa2c21330aba/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:17.496337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf0fe-db57-4ac7-8e8b-fc55eeef5c0d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:17.496337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf0fe-db57-4ac7-8e8b-fc55eeef5c0d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:17.496337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf0fe-db57-4ac7-8e8b-fc55eeef5c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:17.528349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d07cf0fe-db57-4ac7-8e8b-fc55eeef5c0d None None] MFA Rules not processed for user `dd767194597c4aa6ae36fa2c21330aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:17.529616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d07cf0fe-db57-4ac7-8e8b-fc55eeef5c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:17.609945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1177/2354] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:17.620752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:17.622521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:17.715140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27080d31-c204-4200-8fdd-93783859baad admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:17.732743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:17.733565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:17.734001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:17.734391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:17.755535 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:17.758218 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:17.772063 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1178/2355] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:48:22.849903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ef53c-0ab1-4a2f-8774-ada6c91734cb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:22.852172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ef53c-0ab1-4a2f-8774-ada6c91734cb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:22.852172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ef53c-0ab1-4a2f-8774-ada6c91734cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:22.869536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2d5ef53c-0ab1-4a2f-8774-ada6c91734cb None None] MFA Rules not processed for user `9d8116d648d64a62af52d9db6a607e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:22.870755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2d5ef53c-0ab1-4a2f-8774-ada6c91734cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:22.933406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1178/2356] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:27.763824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:32.926160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:39.858397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:39.936662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aef54f-f7e8-43e9-87cc-6e6ffae32b29 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:39.942323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:39.942979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:39.943258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:39.943561 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] PATH_INFO: `/v3/users/dd767194597c4aa6ae36fa2c21330aba` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:39.944827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd767194597c4aa6ae36fa2c21330aba)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:39.946790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:40.134085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd767194597c4aa6ae36fa2c21330aba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:40.134778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd767194597c4aa6ae36fa2c21330aba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:40.136003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd767194597c4aa6ae36fa2c21330aba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:40.262048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd767194597c4aa6ae36fa2c21330aba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:40.270230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd767194597c4aa6ae36fa2c21330aba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:40.291872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1179/2357] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:48:39 2022] DELETE /identity/v3/users/dd767194597c4aa6ae36fa2c21330aba => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:40.298132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:40.298687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:40.299882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:40.327897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:40.329253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:40.393616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1179/2358] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:40.404722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:40.406180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:40.467111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f15533-cd8d-45da-becd-f53d71d09130 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:40.470865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87823f01-a7fb-44bc-85b1-b1b904c6ad43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:40.471573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823f01-a7fb-44bc-85b1-b1b904c6ad43 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:40.471853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823f01-a7fb-44bc-85b1-b1b904c6ad43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:40.472124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823f01-a7fb-44bc-85b1-b1b904c6ad43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:40.494637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87823f01-a7fb-44bc-85b1-b1b904c6ad43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:40.496473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87823f01-a7fb-44bc-85b1-b1b904c6ad43 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:40.527435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1180/2359] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:48:41.030534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:41.032739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:41.052024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7989b-9392-49b8-a21b-bd4750696158 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:41.057745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:41.058608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.059127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.059476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] PATH_INFO: `/v3/projects/104984d8dc7b4055b36b8e6151f2f45f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.060751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=104984d8dc7b4055b36b8e6151f2f45f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:41.064237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:41.228925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104984d8dc7b4055b36b8e6151f2f45f', 'request_id': 'req-4b528837-c003-4648-b0a8-c1f3fe81b57d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:41.309218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104984d8dc7b4055b36b8e6151f2f45f', 'request_id': 'req-4b528837-c003-4648-b0a8-c1f3fe81b57d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:41.319805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104984d8dc7b4055b36b8e6151f2f45f', 'request_id': 'req-4b528837-c003-4648-b0a8-c1f3fe81b57d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:41.333069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] The token cache is being invalidate because project 104984d8dc7b4055b36b8e6151f2f45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:48:41.333361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:41.334020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-4b528837-c003-4648-b0a8-c1f3fe81b57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:41.339814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1180/2360] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:48:41 2022] DELETE /identity/v3/projects/104984d8dc7b4055b36b8e6151f2f45f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:41.341229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.341763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.341862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.363478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:41.364850 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:41.384036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:41.427243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1181/2361] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:48:41.435202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.435489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.435793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.458715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:41.460668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:41.476186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1181/2362] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:41.478490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:41.488754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df909b7-e6bb-46fc-8547-cadb5ad4d32d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:41.495062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:41.495927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.496456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.496874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.497726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:48:41.498192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:41.500054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:41.513825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1182/2363] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:48:41.519082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:41.530460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9934ba-b77a-4bfa-a8f6-b9cb0aca5ad7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:41.538317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:41.538317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.538317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.538317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.538742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:41.540101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:41.688771 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1182/2364] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:48:41 2022] POST /identity/v3/projects => generated 399 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:48:41.690827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:41.697417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:41.699421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:41.702385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ec0a-3dc0-447d-917d-1d160f7af488 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:41.771234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:41.800478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883feb-2d0d-4821-91e3-50aead3a934d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:41.823458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749962-a836-420d-b515-c39f17b815ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:41.823974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749962-a836-420d-b515-c39f17b815ce service placement] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.824213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749962-a836-420d-b515-c39f17b815ce service placement] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.825067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749962-a836-420d-b515-c39f17b815ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.826960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-13749962-a836-420d-b515-c39f17b815ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:41.833106 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:41.834026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.834452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.834872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.835667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:41.839024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:41.843416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13749962-a836-420d-b515-c39f17b815ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:41.845297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13749962-a836-420d-b515-c39f17b815ce service placement] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:41.859592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1183/2365] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:48:41.951443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1183/2366] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:48:41 2022] POST /identity/v3/users => generated 300 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:48:41.958750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749962-a836-420d-b515-c39f17b815ce service placement] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:41.974023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749962-a836-420d-b515-c39f17b815ce service placement] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:41.982706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:41.983201 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:41.983598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:41.984435 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:41.984881 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:42.000865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:42.027756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1184/2367] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:48:42.031393 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:42.058309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959bac6-ad98-4daa-bfa4-075b87751736 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:42.071624 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeede6-576d-4ee7-b1db-f5d00e43a8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:42.073229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeede6-576d-4ee7-b1db-f5d00e43a8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.074024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeede6-576d-4ee7-b1db-f5d00e43a8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.074024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeede6-576d-4ee7-b1db-f5d00e43a8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.074728 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeede6-576d-4ee7-b1db-f5d00e43a8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:42.076484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeede6-576d-4ee7-b1db-f5d00e43a8fd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:42.095085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1184/2368] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:48:42.097079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:42.108408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1091403-7809-4554-82e5-6b8e64cee21c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:42.113446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177d440-a4b9-436d-b4f3-ec0a7c96252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:42.114226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d440-a4b9-436d-b4f3-ec0a7c96252f admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.114664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d440-a4b9-436d-b4f3-ec0a7c96252f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.115039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d440-a4b9-436d-b4f3-ec0a7c96252f admin admin] PATH_INFO: `/v3/projects/c2aefc6464104a6387cd6ec34651c12b/users/c80cc047814c42fc95e6faeafc8e4300/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.137899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177d440-a4b9-436d-b4f3-ec0a7c96252f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2aefc6464104a6387cd6ec34651c12b, user_id=c80cc047814c42fc95e6faeafc8e4300, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:42.142225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177d440-a4b9-436d-b4f3-ec0a7c96252f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:42.222284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1185/2369] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:48:42 2022] PUT /identity/v3/projects/c2aefc6464104a6387cd6ec34651c12b/users/c80cc047814c42fc95e6faeafc8e4300/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:42.224456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.225329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.225658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.293883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] MFA Rules not processed for user `c80cc047814c42fc95e6faeafc8e4300`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:42.295630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:42.393127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677117f7-32f4-4301-b221-50d2b8c7a896 None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.394570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677117f7-32f4-4301-b221-50d2b8c7a896 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.395131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677117f7-32f4-4301-b221-50d2b8c7a896 None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.397192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1186/2370] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:48:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:48:42.411613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.411999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.413001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.436704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1185/2371] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:42.445650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:42.446986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:42.452334 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:42.455707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:42.541620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1187/2372] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 07 14:48:42.554228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:42.556792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:42.562969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35192cb-bb81-4d6f-b3a1-e5eafa6a42cf None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:42.567647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b1f42-9576-4c2b-90b4-c00f2dbaedd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:42.569528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b1f42-9576-4c2b-90b4-c00f2dbaedd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.570675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b1f42-9576-4c2b-90b4-c00f2dbaedd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.571188 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b1f42-9576-4c2b-90b4-c00f2dbaedd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.581178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b1f42-9576-4c2b-90b4-c00f2dbaedd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:42.582889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b1f42-9576-4c2b-90b4-c00f2dbaedd3 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:42.594715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1186/2373] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:48:42.625282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163e4c-78fa-402e-876e-a2771fe7035f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:42.629541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bc01e-3784-43d6-bec4-51d0b52a5a9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:42.630252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc01e-3784-43d6-bec4-51d0b52a5a9f service glance] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.630634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc01e-3784-43d6-bec4-51d0b52a5a9f service glance] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.631847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc01e-3784-43d6-bec4-51d0b52a5a9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.643559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc01e-3784-43d6-bec4-51d0b52a5a9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:42.645363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc01e-3784-43d6-bec4-51d0b52a5a9f service glance] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:42.646601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:42.657431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1188/2374] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:48:42.905021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0897fc6-eed1-46af-a75e-36f147693db7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.905399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0897fc6-eed1-46af-a75e-36f147693db7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.905849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0897fc6-eed1-46af-a75e-36f147693db7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.908783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1187/2375] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:48:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:48:42.929390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad0aec-412f-49db-831f-457be2b41cb4 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:42.929766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad0aec-412f-49db-831f-457be2b41cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:42.930591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad0aec-412f-49db-831f-457be2b41cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:42.952868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-39ad0aec-412f-49db-831f-457be2b41cb4 None None] MFA Rules not processed for user `b50c8a35e28e44bba3a4b4e56057a3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:42.954115 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-39ad0aec-412f-49db-831f-457be2b41cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:43.013887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:43.035879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1189/2376] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:48:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:43.051532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0897fc6-eed1-46af-a75e-36f147693db7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:43.054228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e0897fc6-eed1-46af-a75e-36f147693db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:43.068021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0897fc6-eed1-46af-a75e-36f147693db7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:43.072620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbe663-0642-48d3-805f-e6b12f0a97f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:43.073246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbe663-0642-48d3-805f-e6b12f0a97f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:43.074223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbe663-0642-48d3-805f-e6b12f0a97f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:43.074223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbe663-0642-48d3-805f-e6b12f0a97f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:43.082214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbe663-0642-48d3-805f-e6b12f0a97f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:43.094575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbe663-0642-48d3-805f-e6b12f0a97f9 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:43.094575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1188/2377] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:48:43.392898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274226-84d1-4dfa-b616-7b66747ee47b None None] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:43.393191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274226-84d1-4dfa-b616-7b66747ee47b None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:43.393691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274226-84d1-4dfa-b616-7b66747ee47b None None] PATH_INFO: `/` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:43.394819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1190/2378] 10.222.0.57 () {54 vars in 921 bytes} [Wed Sep 7 14:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 14:48:43.400963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:43.403478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:43.403478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:43.422055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] MFA Rules not processed for user `0cb8a6f42b6e482eb109be9d2f40e64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:43.423302 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:43.433711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1189/2379] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Sep 7 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:43.444432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5274226-84d1-4dfa-b616-7b66747ee47b None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:43.455470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5274226-84d1-4dfa-b616-7b66747ee47b None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:43.459694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7030ac-a05d-4b24-9c3f-97cdf5e55a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:43.460317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7030ac-a05d-4b24-9c3f-97cdf5e55a7d service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:43.460599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7030ac-a05d-4b24-9c3f-97cdf5e55a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:43.460870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7030ac-a05d-4b24-9c3f-97cdf5e55a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:43.470853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7030ac-a05d-4b24-9c3f-97cdf5e55a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:43.472329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7030ac-a05d-4b24-9c3f-97cdf5e55a7d service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:43.483602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1191/2380] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:48:53.416232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=27 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:53.507680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=9 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:53.907230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:54.043649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cac54-f5ce-4f64-b052-12d6c03fca8c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:54.054490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:54.054490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:54.054490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:54.054490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] PATH_INFO: `/v3/users/c80cc047814c42fc95e6faeafc8e4300` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:54.056654 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80cc047814c42fc95e6faeafc8e4300)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:54.060440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:54.293990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80cc047814c42fc95e6faeafc8e4300'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:54.294645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80cc047814c42fc95e6faeafc8e4300'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:54.295260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80cc047814c42fc95e6faeafc8e4300'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:54.369944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80cc047814c42fc95e6faeafc8e4300'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:54.377686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80cc047814c42fc95e6faeafc8e4300'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:54.396835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1190/2381] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:48:53 2022] DELETE /identity/v3/users/c80cc047814c42fc95e6faeafc8e4300 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:54.401438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:54.401837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:54.402181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:54.431217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:54.432474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:54.523841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1192/2382] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:48:54.548375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:54.548375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:54.610051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1911d-ddf6-482d-94f0-3747ad1cf0f7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:54.613811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1477ecc-4dec-4a40-82b9-98ca2eb39845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:54.614477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477ecc-4dec-4a40-82b9-98ca2eb39845 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:54.614758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477ecc-4dec-4a40-82b9-98ca2eb39845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:54.615555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477ecc-4dec-4a40-82b9-98ca2eb39845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:54.625159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477ecc-4dec-4a40-82b9-98ca2eb39845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:54.626590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477ecc-4dec-4a40-82b9-98ca2eb39845 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:54.637685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1191/2383] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:48:55.059567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:48:55.062043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:55.085538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cd4c4-2a77-4b03-af84-9fe0fd2095a2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:48:55.086778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:48:55.087616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:55.087762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:55.088414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] PATH_INFO: `/v3/projects/c2aefc6464104a6387cd6ec34651c12b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:55.089794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2aefc6464104a6387cd6ec34651c12b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:48:55.092252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:48:55.222132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2aefc6464104a6387cd6ec34651c12b', 'request_id': 'req-10106bf3-a580-4980-aac7-057e3f65c9f9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:55.336931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2aefc6464104a6387cd6ec34651c12b', 'request_id': 'req-10106bf3-a580-4980-aac7-057e3f65c9f9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:55.344983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2aefc6464104a6387cd6ec34651c12b', 'request_id': 'req-10106bf3-a580-4980-aac7-057e3f65c9f9'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:55.359250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] The token cache is being invalidate because project c2aefc6464104a6387cd6ec34651c12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:48:55.359614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:55.360403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-10106bf3-a580-4980-aac7-057e3f65c9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:48:55.362249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1193/2384] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:48:55 2022] DELETE /identity/v3/projects/c2aefc6464104a6387cd6ec34651c12b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:48:58.475361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:48:58.475873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:48:58.475982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:48:58.495971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] MFA Rules not processed for user `cfd87fa14f924cb897959950c79739ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:48:58.497532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:48:58.518029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:48:58.555866 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1192/2385] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:48:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:01.660996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58749d4a-8557-4e3c-b186-e92795b86347 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:01.661981 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58749d4a-8557-4e3c-b186-e92795b86347 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:01.662082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58749d4a-8557-4e3c-b186-e92795b86347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:01.682380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-58749d4a-8557-4e3c-b186-e92795b86347 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:01.683623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-58749d4a-8557-4e3c-b186-e92795b86347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:01.737376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1194/2386] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:49:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:01.751167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:01.752679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:01.832113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70de6-92ee-426b-ad29-7f0c73abc2d2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:01.837327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:01.837945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:01.838355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:01.838637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:01.847215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:01.850158 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:01.858497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1193/2387] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:04.014369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d4f97-a484-427e-9a43-be46ac196583 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:04.015467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d4f97-a484-427e-9a43-be46ac196583 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:04.015933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d4f97-a484-427e-9a43-be46ac196583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:04.038601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d19d4f97-a484-427e-9a43-be46ac196583 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:04.040243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d19d4f97-a484-427e-9a43-be46ac196583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:04.058147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1195/2388] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:49:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:49:04.091519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:04.112083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a8fb-8064-459f-b6f6-39ed8f8a6274 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:04.128506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:04.132314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:04.132314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:04.132314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:04.147122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:04.148969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:04.167524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1194/2389] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:14.032093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:14.164311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=53 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:18.240654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9df17-1ff3-4aca-9ac1-cc6767ec13ce None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:18.241252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9df17-1ff3-4aca-9ac1-cc6767ec13ce None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:18.241364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9df17-1ff3-4aca-9ac1-cc6767ec13ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:18.264538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-acf9df17-1ff3-4aca-9ac1-cc6767ec13ce None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:18.266648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-acf9df17-1ff3-4aca-9ac1-cc6767ec13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:18.292191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1196/2390] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:49:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:18.309434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:18.322893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a495d-cba6-4a53-b27d-f7e31b454a97 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:18.328809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:18.329643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:18.330176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:18.330451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:18.342126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:18.343864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:18.355570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1195/2391] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:19.649913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d226e-e34b-403f-b704-4b83de8ace94 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:19.650434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d226e-e34b-403f-b704-4b83de8ace94 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:19.650522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d226e-e34b-403f-b704-4b83de8ace94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:19.670861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8c0d226e-e34b-403f-b704-4b83de8ace94 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:19.672095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8c0d226e-e34b-403f-b704-4b83de8ace94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:19.681061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1197/2392] 10.222.0.57 () {58 vars in 967 bytes} [Wed Sep 7 14:49:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:19.716780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:19.735874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9aeb-09b8-4e85-9c19-992722920d67 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:19.742412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:19.743260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:19.743613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:19.743967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:19.753622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:19.755694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:19.765856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1196/2393] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:20.546389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20ebba-ff2a-4e0a-a62a-cc07f7e65ec5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:20.546841 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20ebba-ff2a-4e0a-a62a-cc07f7e65ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:20.547127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20ebba-ff2a-4e0a-a62a-cc07f7e65ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:20.574006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-3c20ebba-ff2a-4e0a-a62a-cc07f7e65ec5 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:20.575837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3c20ebba-ff2a-4e0a-a62a-cc07f7e65ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:20.583873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1198/2394] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:49:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:20.600664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:20.612687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c8c35-d576-488e-98d6-f930124479a5 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:20.619331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:20.619842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:20.620339 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:20.620788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:20.633954 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:20.636069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:20.653202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1197/2395] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:21.376171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:21.376835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:21.376835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:21.399478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:21.400703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:21.408676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1199/2396] 10.222.0.57 () {58 vars in 967 bytes} [Wed Sep 7 14:49:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:21.433542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:21.446057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade889b1-4bc3-4bf3-9a6e-09b73445002f service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:21.452892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323966d-e909-4f2c-94be-b486a69f3bea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:21.453832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323966d-e909-4f2c-94be-b486a69f3bea service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:21.454384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323966d-e909-4f2c-94be-b486a69f3bea service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:21.454949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323966d-e909-4f2c-94be-b486a69f3bea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:21.468969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323966d-e909-4f2c-94be-b486a69f3bea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:21.471793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323966d-e909-4f2c-94be-b486a69f3bea service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:21.484844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1198/2397] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:24.980005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:24.981514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:25.046996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3a67c-3ec8-4049-b381-b69db7420efb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:25.052483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:25.053142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:25.053421 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:25.053692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] PATH_INFO: `/v3/users/133ae50f4b814e978c875bf61a9db611` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:25.054840 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=133ae50f4b814e978c875bf61a9db611)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:25.056822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:25.266205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133ae50f4b814e978c875bf61a9db611'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:25.335781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133ae50f4b814e978c875bf61a9db611'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:25.343922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133ae50f4b814e978c875bf61a9db611'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:25.345215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133ae50f4b814e978c875bf61a9db611'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:25.346032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133ae50f4b814e978c875bf61a9db611'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:25.365484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1200/2398] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:49:24 2022] DELETE /identity/v3/users/133ae50f4b814e978c875bf61a9db611 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:25.370192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d708c-af87-4643-bfcd-b4307dc5d005 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:25.370697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d708c-af87-4643-bfcd-b4307dc5d005 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:25.370794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d708c-af87-4643-bfcd-b4307dc5d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:25.389597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3a4d708c-af87-4643-bfcd-b4307dc5d005 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:25.390788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3a4d708c-af87-4643-bfcd-b4307dc5d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:25.450879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1199/2399] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:25.461459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:25.463171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:25.523730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:25.523953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:25.524246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:25.540056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523072ae-c2f5-4a11-89df-c033a5f795b7 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:25.546370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:25.547079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:25.547598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:25.547722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:25.559319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:25.561016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:25.562768 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:25.563710 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:25.565499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:25.577316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1201/2400] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:49:25.654603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1200/2401] 10.222.0.57 () {58 vars in 967 bytes} [Wed Sep 7 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:25.671134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:25.672649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:25.779414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c794ba6-bafc-4fb4-badb-8b2f63633c53 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:25.785694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d6c7-fb8e-4598-b9d2-9646e28193ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:25.786451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d6c7-fb8e-4598-b9d2-9646e28193ba service swift] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:25.787036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d6c7-fb8e-4598-b9d2-9646e28193ba service swift] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:25.787036 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d6c7-fb8e-4598-b9d2-9646e28193ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:25.797593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d6c7-fb8e-4598-b9d2-9646e28193ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:25.803692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d6c7-fb8e-4598-b9d2-9646e28193ba service swift] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:25.823183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1202/2402] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:25.946049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:25.949439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:25.970773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0505a34-d152-49bc-b623-291c963e3991 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:25.982683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:25.983354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:25.983634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:25.984051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] PATH_INFO: `/v3/projects/c7e0ace10fe241aba9f2ab06d05b0032` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:25.985255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e0ace10fe241aba9f2ab06d05b0032)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:25.987485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:26.137457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:26.139208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e0ace10fe241aba9f2ab06d05b0032', 'request_id': 'req-ae9922a7-3c79-41a5-8ba0-3286b899e13e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:26.217385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e0ace10fe241aba9f2ab06d05b0032', 'request_id': 'req-ae9922a7-3c79-41a5-8ba0-3286b899e13e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:26.225434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e0ace10fe241aba9f2ab06d05b0032', 'request_id': 'req-ae9922a7-3c79-41a5-8ba0-3286b899e13e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:26.237751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] The token cache is being invalidate because project c7e0ace10fe241aba9f2ab06d05b0032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:49:26.238282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:26.238919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ae9922a7-3c79-41a5-8ba0-3286b899e13e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:26.241021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1201/2403] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:49:25 2022] DELETE /identity/v3/projects/c7e0ace10fe241aba9f2ab06d05b0032 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:26.248072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.249409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.249936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.280770 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:26.282541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:26.351364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1203/2404] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:26.359126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.359412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.359687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.384673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:26.385920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:26.394404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1202/2405] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:26.399935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:26.413099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de64a78-303c-4b17-947a-d9df8879d215 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:26.418753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:26.419202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.419406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.419711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.420399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:49:26.421670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:26.422689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:26.435496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1204/2406] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:49:26.439931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:26.450242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a68b59-da20-4349-9d1c-e72d132cce48 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:26.455921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:26.456825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.457277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.457704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.459095 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:26.461340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:26.556437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1203/2407] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:49:26 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:49:26.560860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:26.571616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e121-db92-4f28-88a6-deae6fae2fb4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:26.637325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:26.637929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.638205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.638568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.639233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:26.641205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:26.788033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1205/2408] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:49:26 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:49:26.792233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:26.808067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffc147-c31e-4f82-b6c9-2d95bd2337af admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:26.812665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:26.813351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.813869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.814214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.815485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:26.818326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:26.829992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1204/2409] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:49:26.833844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:26.839754 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:26.846872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bebb6c-9115-4097-a844-edabdb5ceb0a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:26.854571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eed3a-a8be-40ad-8ca1-d546b819e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:26.855165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eed3a-a8be-40ad-8ca1-d546b819e4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.856310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eed3a-a8be-40ad-8ca1-d546b819e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.856588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eed3a-a8be-40ad-8ca1-d546b819e4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.857234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eed3a-a8be-40ad-8ca1-d546b819e4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:26.858970 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eed3a-a8be-40ad-8ca1-d546b819e4d5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:26.870808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1206/2410] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:49:26.874996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:26.886371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c963c-dd32-4891-ab6b-6a154d79919f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:26.891023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:26.891964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:26.891964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:26.894079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] PATH_INFO: `/v3/projects/5c7b26a4966c4c309d058d1d19d5d553/users/2a6a637072b04667a3f21da9c57325c0/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:26.908725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7b26a4966c4c309d058d1d19d5d553, user_id=2a6a637072b04667a3f21da9c57325c0, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:26.911502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:27.013539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1205/2411] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:49:26 2022] PUT /identity/v3/projects/5c7b26a4966c4c309d058d1d19d5d553/users/2a6a637072b04667a3f21da9c57325c0/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:27.018933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:27.019442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:27.019821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:27.042850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:27.044301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:27.144963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1207/2412] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:27.154683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:27.156511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:27.214414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce1438-59f2-4576-812d-7fac6cfdc055 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:27.218023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:27.218629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:27.218907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:27.219178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:27.228576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:27.230041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:27.240273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1206/2413] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:49:29.263954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:29.266001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:29.280428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a2a8-8f19-40d8-baeb-f2250424599d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:29.286029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:29.286895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:29.287396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:29.287859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] PATH_INFO: `/v3/users/0fb67f4c58934a568fedf74dc9d45c8d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:29.289279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb67f4c58934a568fedf74dc9d45c8d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:29.291449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:29.467848 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb67f4c58934a568fedf74dc9d45c8d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:29.545046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb67f4c58934a568fedf74dc9d45c8d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:29.554484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb67f4c58934a568fedf74dc9d45c8d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:29.555655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb67f4c58934a568fedf74dc9d45c8d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:29.556219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e2fefbb4-6747-4a21-959c-5bef2658b7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb67f4c58934a568fedf74dc9d45c8d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:29.577099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1208/2414] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:49:29 2022] DELETE /identity/v3/users/0fb67f4c58934a568fedf74dc9d45c8d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:29.825845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:29.837713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cef9f-7f31-494d-a1bf-7d3eab79c638 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:29.917890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:29.918600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:29.918873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:29.919146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] PATH_INFO: `/v3/projects/2497a9df3f354a5789cc9f7735e39828` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:29.920580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] RBAC: Authorizing `identity:delete_project(project_id=2497a9df3f354a5789cc9f7735e39828)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:29.922565 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:29.945357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:30.037947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2497a9df3f354a5789cc9f7735e39828', 'request_id': 'req-34b4d4f2-fc42-43f5-ab12-620ffad342de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:30.092203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:30.092491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:30.092969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:30.119278 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] MFA Rules not processed for user `8eb5509df78d45acbebc332a889251dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:30.129150 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:30.133914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2497a9df3f354a5789cc9f7735e39828', 'request_id': 'req-34b4d4f2-fc42-43f5-ab12-620ffad342de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:30.140945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2497a9df3f354a5789cc9f7735e39828', 'request_id': 'req-34b4d4f2-fc42-43f5-ab12-620ffad342de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:30.155582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] The token cache is being invalidate because project 2497a9df3f354a5789cc9f7735e39828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:49:30.155879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:30.157382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-34b4d4f2-fc42-43f5-ab12-620ffad342de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:49:30.166170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1207/2415] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:49:29 2022] DELETE /identity/v3/projects/2497a9df3f354a5789cc9f7735e39828 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:30.227326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1209/2416] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:49:30.378733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c0fb0-df48-46b6-965d-e4e9c74d414e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:30.379273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c0fb0-df48-46b6-965d-e4e9c74d414e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:30.380398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c0fb0-df48-46b6-965d-e4e9c74d414e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:30.419992 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4f2c0fb0-df48-46b6-965d-e4e9c74d414e None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:30.421767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4f2c0fb0-df48-46b6-965d-e4e9c74d414e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:30.522870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1208/2417] 10.222.0.57 () {58 vars in 967 bytes} [Wed Sep 7 14:49:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:30.540676 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:30.542746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:30.583635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:30.682591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ba3fd-9709-4d54-ba43-303612b97cd3 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:30.695039 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:30.695933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:30.696204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:30.696546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:30.712059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:30.714586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:30.726691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1210/2418] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:49:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:49:34.109309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.109824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.110015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.141176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] MFA Rules not processed for user `2a6a637072b04667a3f21da9c57325c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:34.142822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:34.214083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1209/2419] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:34.218929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:34.220399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:34.288595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e9037-50eb-43d8-b995-742470f88800 service swift] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:34.293165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:34.293781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.294064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.294337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.294983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:34.297185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:34.421515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1211/2420] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:49:34 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:49:34.424622 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:34.437613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13cf3d-6d27-489e-94db-baafed1baeb0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:34.497090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:34.527269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:34.528073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.528488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.528860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.530195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:34.532623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:34.589183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1210/2421] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:49:34 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:49:34.593192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:34.603613 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68fe96-a2e7-4961-beb1-9009bacfbf29 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:34.608376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:34.612581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.612581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.612581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.612581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:34.612581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:34.624202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1212/2422] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:49:34.628112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:34.639635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daac9a6-d05c-4d08-bed7-45439d55f98c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:34.645054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:34.646249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.646526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.646799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] PATH_INFO: `/v3/projects/03f993468f1842309b2f338ed903d1d5/users/725b741992fe4c39939e2f29a72a3ab1/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.662335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f993468f1842309b2f338ed903d1d5, user_id=725b741992fe4c39939e2f29a72a3ab1, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:34.665057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:34.758443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1211/2423] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:49:34 2022] PUT /identity/v3/projects/03f993468f1842309b2f338ed903d1d5/users/725b741992fe4c39939e2f29a72a3ab1/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:34.760068 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:34.771531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e17d2-da5f-4b3c-b0c4-ba386e6cc93c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:34.839108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9980b-bc31-4624-ace0-b7b56030bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:34.839650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9980b-bc31-4624-ace0-b7b56030bc97 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.839912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9980b-bc31-4624-ace0-b7b56030bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.840330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9980b-bc31-4624-ace0-b7b56030bc97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.840963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9980b-bc31-4624-ace0-b7b56030bc97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:34.842665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9980b-bc31-4624-ace0-b7b56030bc97 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:34.853863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1213/2424] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:49:34.857892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:34.868976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f61f7-844f-48b2-bb84-4594ac7f7e10 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:34.873716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:34.874442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:34.874653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:34.874918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] PATH_INFO: `/v3/projects/03f993468f1842309b2f338ed903d1d5/users/725b741992fe4c39939e2f29a72a3ab1/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:34.876833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f993468f1842309b2f338ed903d1d5, user_id=725b741992fe4c39939e2f29a72a3ab1, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:34.879466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:34.891026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1212/2425] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:49:34 2022] PUT /identity/v3/projects/03f993468f1842309b2f338ed903d1d5/users/725b741992fe4c39939e2f29a72a3ab1/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:49:41.375282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b11baa-2c14-4980-b82d-9348efdb08e2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:41.376892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b11baa-2c14-4980-b82d-9348efdb08e2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:41.377565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b11baa-2c14-4980-b82d-9348efdb08e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:41.414897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:41.436880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d2b11baa-2c14-4980-b82d-9348efdb08e2 None None] MFA Rules not processed for user `725b741992fe4c39939e2f29a72a3ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:49:41.438621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d2b11baa-2c14-4980-b82d-9348efdb08e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:41.524108 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1214/2426] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:49:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:49:41.538375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:49:41.540460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:49:41.636532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51ede-46f5-474b-b8a8-540672e79992 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:49:41.646601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60366b-d133-49ed-b620-12784b991bec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:49:41.647241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60366b-d133-49ed-b620-12784b991bec service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:49:41.647519 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60366b-d133-49ed-b620-12784b991bec service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:49:41.647796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60366b-d133-49ed-b620-12784b991bec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:49:41.733737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60366b-d133-49ed-b620-12784b991bec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:49:41.735434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60366b-d133-49ed-b620-12784b991bec service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:49:41.744877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1213/2427] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:49:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:49:51.525209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=30 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:49:51.741201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:05.458441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:05.458968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:05.459187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:05.478844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] MFA Rules not processed for user `2a6a637072b04667a3f21da9c57325c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:05.480032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:05.487016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1215/2428] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:15.472809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:18.076927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:18.077460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:18.077806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:18.099265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] MFA Rules not processed for user `9d8116d648d64a62af52d9db6a607e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:18.100508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:18.156469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1214/2429] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:26.170523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:26.173656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:26.239315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef7e26-9526-47a9-83d9-45ab7aed9936 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:26.244935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:26.245551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:26.245909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:26.246187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] PATH_INFO: `/v3/users/cfd87fa14f924cb897959950c79739ef` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:26.247290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd87fa14f924cb897959950c79739ef)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:26.249358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:26.451345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd87fa14f924cb897959950c79739ef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:26.523683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd87fa14f924cb897959950c79739ef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:26.532259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd87fa14f924cb897959950c79739ef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:26.532908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd87fa14f924cb897959950c79739ef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:26.533545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6819f9e9-e238-40cd-8615-52e98b395b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd87fa14f924cb897959950c79739ef'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:26.552400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1216/2430] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:50:26 2022] DELETE /identity/v3/users/cfd87fa14f924cb897959950c79739ef => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:50:26.812831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:26.826863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb545e51-9909-4992-acc5-2922effe6203 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:26.926018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:26.926672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:26.926950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:26.927224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] PATH_INFO: `/v3/projects/2ed99ce2509c4163b314baa4cf3e5ae5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:26.928397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed99ce2509c4163b314baa4cf3e5ae5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:26.930304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.040646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed99ce2509c4163b314baa4cf3e5ae5', 'request_id': 'req-6e4226ad-f747-4048-ae9b-187b15229e7d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:27.114005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed99ce2509c4163b314baa4cf3e5ae5', 'request_id': 'req-6e4226ad-f747-4048-ae9b-187b15229e7d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:27.119922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed99ce2509c4163b314baa4cf3e5ae5', 'request_id': 'req-6e4226ad-f747-4048-ae9b-187b15229e7d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:27.132105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] The token cache is being invalidate because project 2ed99ce2509c4163b314baa4cf3e5ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:50:27.132516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:27.133172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6e4226ad-f747-4048-ae9b-187b15229e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:50:27.135213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1215/2431] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:50:26 2022] DELETE /identity/v3/projects/2ed99ce2509c4163b314baa4cf3e5ae5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:50:27.140476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.141180 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.141531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.164283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:27.165516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:27.226309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1217/2432] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:27.234455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.234797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.235077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.255630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:27.256934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:27.266709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1216/2433] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:50:27.271179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.281260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b23569-eae2-4a0b-a02d-c8185f33b1d5 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:27.286007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:27.286584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.286868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.287142 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.287786 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:50:27.288090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:27.289733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.301417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1218/2434] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:50:27.311505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.320839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc18121-4050-449c-a4b5-16151409df64 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:27.326178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:27.326783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.327060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.327329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.327958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:27.329884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.434675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1217/2435] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:50:27 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:50:27.438930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.449951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfc2d1-8fc6-4923-9cd5-1ec5370c8c7d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:27.453706 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:27.510537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:27.510537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.510537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.510537 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.512293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:27.512862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.625356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1219/2436] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:50:27 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:50:27.629419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.641579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfa28e-8760-4b0f-bb29-de1ffc779be2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:27.648429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:27.649737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.649998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.650423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.651720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:27.654143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.657462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:27.666608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1218/2437] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:50:27.670417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.680808 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6e91-cb5b-4b68-a62b-b7989bf4a3e0 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:27.685614 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6065d7-4dbf-45f1-8b66-0affed45ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:27.686207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6065d7-4dbf-45f1-8b66-0affed45ee41 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.686483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6065d7-4dbf-45f1-8b66-0affed45ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.686752 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6065d7-4dbf-45f1-8b66-0affed45ee41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.687382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6065d7-4dbf-45f1-8b66-0affed45ee41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:27.689009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6065d7-4dbf-45f1-8b66-0affed45ee41 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.700593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1220/2438] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:50:27.706587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.718234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee2ce-1ec1-4a93-8f56-ae728c23c55a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:27.723728 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:27.724427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.724709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.725024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] PATH_INFO: `/v3/projects/6bc6eee4ac194763a27bb87522d47c6c/users/2d34e301934743d5a457ae7fd979a53f/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.740845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc6eee4ac194763a27bb87522d47c6c, user_id=2d34e301934743d5a457ae7fd979a53f, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:27.743503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:27.803846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1219/2439] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:50:27 2022] PUT /identity/v3/projects/6bc6eee4ac194763a27bb87522d47c6c/users/2d34e301934743d5a457ae7fd979a53f/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:50:27.823611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:27.823992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:27.824352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:27.843959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:27.845264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:27.913064 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1221/2440] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:27.933959 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:27.935776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:27.996442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edec44-2951-4120-9795-991dd84cfbcf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:28.000036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24605d4-4c19-40d4-bb44-0ce7ab18dc5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:28.005826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24605d4-4c19-40d4-bb44-0ce7ab18dc5d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:28.005826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24605d4-4c19-40d4-bb44-0ce7ab18dc5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:28.005826 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24605d4-4c19-40d4-bb44-0ce7ab18dc5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:28.012055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24605d4-4c19-40d4-bb44-0ce7ab18dc5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:28.013631 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24605d4-4c19-40d4-bb44-0ce7ab18dc5d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:28.023945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1220/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:50:30.938634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:30.940359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:30.956538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef3eaf-9118-40de-ad9b-0a0714de8491 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:30.956909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:30.957559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:30.957839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:30.958113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:30.959865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:31.046562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:31.048986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:31.060915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1222/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:50:34.856028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:34.866090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:34.866090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:34.893875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] MFA Rules not processed for user `2d34e301934743d5a457ae7fd979a53f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:34.893875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:34.988298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1221/2443] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:34.991264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:35.002773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bdc19-e2ac-4dab-8479-102e59793eaf service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:35.007643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:35.008306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:35.008583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:35.008851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:35.009483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:35.011944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:35.119727 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:35.134346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1223/2444] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:50:34 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:50:35.135746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:35.147831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc44a5-76ef-48c8-96f4-733624bb7e81 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:35.234184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:35.234813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:35.235088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:35.235358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:35.236001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:35.238125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:35.258146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:35.342075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1222/2445] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:50:35 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:50:35.346070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:35.357441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782fc26-9480-465f-bc5c-b1e7bf8a05d2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:35.367892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:35.367892 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:35.368558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:35.368558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:35.369692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:35.371793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:35.392927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1224/2446] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:50:35.394210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:35.424688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e334c-a63c-48ad-870d-c7e19d393a4c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:35.432278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:35.432955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:35.433234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:35.433510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] PATH_INFO: `/v3/projects/35a25066d1b4476a91bd4777b4bffb63/users/207052f6f3b746e3921e3c0cc2d902ff/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:35.450389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a25066d1b4476a91bd4777b4bffb63, user_id=207052f6f3b746e3921e3c0cc2d902ff, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:35.453791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:35.554814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1223/2447] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:50:35 2022] PUT /identity/v3/projects/35a25066d1b4476a91bd4777b4bffb63/users/207052f6f3b746e3921e3c0cc2d902ff/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:50:35.559602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:35.573242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfafb95-aa1f-4ebf-b1ea-a8b6a7289486 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:35.682926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f06ce3-ad88-4155-ae1a-708536bfb3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:35.683616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f06ce3-ad88-4155-ae1a-708536bfb3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:35.683896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f06ce3-ad88-4155-ae1a-708536bfb3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:35.684186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f06ce3-ad88-4155-ae1a-708536bfb3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:35.684875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f06ce3-ad88-4155-ae1a-708536bfb3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:35.686772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f06ce3-ad88-4155-ae1a-708536bfb3f6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:35.698296 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1225/2448] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:50:35.702555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:35.714107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836cd8b-a628-4e9f-9d74-8a0fc01e764f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:35.719478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb9dc-51d0-4a94-be43-938fbd0f2042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:35.720187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb9dc-51d0-4a94-be43-938fbd0f2042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:35.720475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb9dc-51d0-4a94-be43-938fbd0f2042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:35.720747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb9dc-51d0-4a94-be43-938fbd0f2042 admin admin] PATH_INFO: `/v3/projects/35a25066d1b4476a91bd4777b4bffb63/users/207052f6f3b746e3921e3c0cc2d902ff/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:35.722724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb9dc-51d0-4a94-be43-938fbd0f2042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a25066d1b4476a91bd4777b4bffb63, user_id=207052f6f3b746e3921e3c0cc2d902ff, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:35.726298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb9dc-51d0-4a94-be43-938fbd0f2042 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:35.738918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1224/2449] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:50:35 2022] PUT /identity/v3/projects/35a25066d1b4476a91bd4777b4bffb63/users/207052f6f3b746e3921e3c0cc2d902ff/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:50:36.487348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:36.487823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:36.487917 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:36.507047 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] MFA Rules not processed for user `eeab924bf7a646779b48f57f14710e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:36.508303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:36.569609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1226/2450] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:42.048019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08165bdb-c63e-49a3-ae29-20763085f9a1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:42.048524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08165bdb-c63e-49a3-ae29-20763085f9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:42.048723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08165bdb-c63e-49a3-ae29-20763085f9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:42.089550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-08165bdb-c63e-49a3-ae29-20763085f9a1 None None] MFA Rules not processed for user `207052f6f3b746e3921e3c0cc2d902ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:42.091161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-08165bdb-c63e-49a3-ae29-20763085f9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:42.196825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1225/2451] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:50:42.211077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:42.213311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:42.310969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5b565-0114-44ef-a1ef-dce30c1428f2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:42.317403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:42.318374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:42.318730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:42.319010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:42.348009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:50:42.400647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:42.400647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:42.405748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1227/2452] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:50:44.950241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:44.950740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:44.952258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:44.972517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] MFA Rules not processed for user `2d34e301934743d5a457ae7fd979a53f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:44.979781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:44.989453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:44.990897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:44.996643 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1226/2453] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:50:45.075404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572d7a7-4ff0-49ed-ad61-07668617caad service nova] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:45.079206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:45.079919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:45.080424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:45.081812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:45.083784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:45.163314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:45.164873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:45.176919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1228/2454] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:50:49.450209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:49.463409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdff0c-de79-42e5-8185-d27539abc5b3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:49.469673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:49.470286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:49.470587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:49.470856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:49.472604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:49.483734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:49.485332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:49.494695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1227/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:50:51.745720 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:51.803536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665bfc-6d6f-4ff9-ab99-f2f7830a4ea9 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:51.807159 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:51.808723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:51.808723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:51.808932 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:51.810655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:51.825492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:51.826940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5395a51-f0ec-411a-b9fb-cca42980904e service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:51.838450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1229/2456] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:50:54.376722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:50:54.389876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7995c-379d-434e-8445-8cf9ec530002 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:50:54.393607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:50:54.394487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:54.394761 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:54.395032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:54.399640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:54.428328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:50:54.428328 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:50:54.434515 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1228/2457] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:50:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:50:59.705878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:50:59.706397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:50:59.706558 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:50:59.726553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] MFA Rules not processed for user `8eb5509df78d45acbebc332a889251dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:50:59.727746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:50:59.734628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1230/2458] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:50:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:51:00.038391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:00.047420 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7178-fe00-4ee3-90e6-3c9b53eba84a service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:00.051144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:00.051729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:00.052000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:00.052296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:00.053990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:00.063762 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:00.065202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:00.073277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1229/2459] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:00.467123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:00.468555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:00.489778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:00.542497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29def25-6eda-4cee-9560-11612e4248a2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:00.548132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:00.548763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:00.549041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:00.549311 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:00.551034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:00.562184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:00.563764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:00.573338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1231/2460] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:04.383898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:04.385488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:04.400465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3808-0b78-4226-8a02-e774594447d0 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:04.404784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:04.405819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:04.406335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:04.406836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:04.409010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:04.491552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:04.493349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:04.494803 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:04.505601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1230/2461] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:05.113699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:05.125416 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e362a-8879-4d58-bff5-af241fd09e78 service placement] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:05.130040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:05.130930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:05.131325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:05.131693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:05.133997 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:05.145875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:05.147393 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:05.156940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1232/2462] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:12.659236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:12.674600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cca26c-3cba-4b3e-a015-06ffabdf2c35 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:12.680702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:12.681418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:12.681962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:12.682244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:12.684392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:12.697963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:12.700483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:12.712539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1231/2463] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:13.207546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:13.209464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:13.247636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd5f40-dea5-4db5-a757-24824c614866 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:13.257524 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:13.259662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:13.260199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:13.260701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:13.263861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:13.282392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:13.284853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:13.298850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1233/2464] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:22.707265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=9 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:23.292925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=37 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:25.010557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:25.022738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e0dbf-8976-4ca3-95d8-b1a251adb512 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:25.029018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:25.029874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:25.030162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:25.030437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:25.032383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:25.049117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:25.050939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:25.067144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1232/2465] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:29.454572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:29.464314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd3ee9-1aab-4b85-97ed-a6fee3ce6cb7 service nova] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:29.468468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:29.469058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:29.469335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:29.469607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:29.478345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:29.479765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:29.489723 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1234/2466] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:30.854511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:30.864727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc551f-89f1-4ad4-b52e-4e096a4f7fc4 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:30.868544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:30.869222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:30.869508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:30.869784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:30.871501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:30.938608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:30.940031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:30.950123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1233/2467] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:34.846890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:34.848499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:34.863916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58285c0b-f53d-4c5c-b7b6-76c73f7cd0ea service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:34.868472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:34.869098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:34.869377 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:34.869710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] PATH_INFO: `/v3/users/9d8116d648d64a62af52d9db6a607e13` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:34.870842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8116d648d64a62af52d9db6a607e13)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:34.872832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:35.058053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8116d648d64a62af52d9db6a607e13'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.134943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8116d648d64a62af52d9db6a607e13'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.142856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8116d648d64a62af52d9db6a607e13'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.143479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8116d648d64a62af52d9db6a607e13'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.144093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6436076c-2f08-43ec-850d-557c804b143a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8116d648d64a62af52d9db6a607e13'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.164413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1235/2468] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:51:34 2022] DELETE /identity/v3/users/9d8116d648d64a62af52d9db6a607e13 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:51:35.401375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:35.411904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84391-1d7e-44aa-8b1f-78f32d015994 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:35.470732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:35.471361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:35.471636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:35.471905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] PATH_INFO: `/v3/projects/6869744faba14c0bb69be840cea600f0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:35.473044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6869744faba14c0bb69be840cea600f0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:35.474940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:35.571831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6869744faba14c0bb69be840cea600f0', 'request_id': 'req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.643043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6869744faba14c0bb69be840cea600f0', 'request_id': 'req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.649322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6869744faba14c0bb69be840cea600f0', 'request_id': 'req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.660523 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] The token cache is being invalidate because project 6869744faba14c0bb69be840cea600f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:51:35.660808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.661437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-370d0c5c-5831-4bb4-87e8-8ae2a9137c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:51:35.662951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1234/2469] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:51:35 2022] DELETE /identity/v3/projects/6869744faba14c0bb69be840cea600f0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:51:35.667943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:35.668267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:35.668541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:35.688353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:51:35.689556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:35.747745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1236/2470] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:51:35.755514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:35.755813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:35.759836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:35.776892 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:51:35.778092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:35.785793 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1235/2471] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:51:35.789984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:35.800020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46b6b-7cf6-4382-a01a-7bbfa4f0c0cd None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:35.804597 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:35.805188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:35.805466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:35.805742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:35.806383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:51:35.806684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:35.808314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:35.820425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1237/2472] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:51:35.824187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:35.830084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:35.833473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d71ae7-b1b3-4dff-89cd-ce43f55fb27a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:35.838204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:35.838796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:35.839068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:35.839332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:35.839945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:35.841869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:35.931007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1236/2473] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:51:35 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:51:35.935555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:35.945675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26ab4c-14d2-4832-8d33-c001f8ea6749 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:35.992908 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:36.006380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:36.006985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:36.007264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:36.007535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:36.008239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:36.010240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:36.100053 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1238/2474] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:51:35 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:51:36.104346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:36.115560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f928d-7fce-4ec9-b615-fdf26beccb7e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:36.120365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:36.120969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:36.121246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:36.121518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:36.122780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:36.124432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:36.134215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1237/2475] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:51:36.137996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:36.147978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0967cd-80b2-4e1c-8443-f332ade7a30c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:36.152659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd180c5-ec57-4bfc-a977-7dc8fc112ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:36.153775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd180c5-ec57-4bfc-a977-7dc8fc112ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:36.154050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd180c5-ec57-4bfc-a977-7dc8fc112ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:36.154316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd180c5-ec57-4bfc-a977-7dc8fc112ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:36.154945 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd180c5-ec57-4bfc-a977-7dc8fc112ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:36.156572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd180c5-ec57-4bfc-a977-7dc8fc112ea0 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:36.167704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1239/2476] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:51:36.172272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:36.182239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a81a02-8508-4f6d-b7e4-2cc3c17d6c9a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:36.186800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:36.187440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:36.187715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:36.187985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] PATH_INFO: `/v3/projects/375694ab50a34237bf53010138421f1b/users/6e21a071d561423689653593b30cdaed/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:36.202274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375694ab50a34237bf53010138421f1b, user_id=6e21a071d561423689653593b30cdaed, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:36.204962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:36.271673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1238/2477] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:51:36 2022] PUT /identity/v3/projects/375694ab50a34237bf53010138421f1b/users/6e21a071d561423689653593b30cdaed/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:51:36.276759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de3efb-b76e-4a78-b0a6-e8a5c24a3579 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:36.277034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de3efb-b76e-4a78-b0a6-e8a5c24a3579 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:36.277335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de3efb-b76e-4a78-b0a6-e8a5c24a3579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:36.297092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-21de3efb-b76e-4a78-b0a6-e8a5c24a3579 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:51:36.298293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-21de3efb-b76e-4a78-b0a6-e8a5c24a3579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:36.355011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1240/2478] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:51:36.365345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:36.367087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:36.424237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11f7a-fc4e-4d5c-8ed0-559739b29688 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:36.427929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:36.428577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:36.428858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:36.429145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:36.438661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:36.440055 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:36.450689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1239/2479] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:41.402272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7510f-f713-414d-8311-7f5411ad0756 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:41.402837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7510f-f713-414d-8311-7f5411ad0756 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:41.402837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7510f-f713-414d-8311-7f5411ad0756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:41.432973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-daa7510f-f713-414d-8311-7f5411ad0756 None None] MFA Rules not processed for user `6e21a071d561423689653593b30cdaed`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:51:41.434203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-daa7510f-f713-414d-8311-7f5411ad0756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:41.546584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1241/2480] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:51:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:51:41.562527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:41.564226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:41.643762 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f63df-22c6-4f24-8b64-5341646c9726 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:41.649776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5c4e-6e4e-48b5-9ca2-f048dccea564 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:41.650397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5c4e-6e4e-48b5-9ca2-f048dccea564 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:41.650679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5c4e-6e4e-48b5-9ca2-f048dccea564 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:41.650953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5c4e-6e4e-48b5-9ca2-f048dccea564 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:41.660585 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5c4e-6e4e-48b5-9ca2-f048dccea564 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:41.662185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5c4e-6e4e-48b5-9ca2-f048dccea564 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:41.672616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1240/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:51:44.137409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:44.138305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:44.138575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:44.158709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] MFA Rules not processed for user `6e21a071d561423689653593b30cdaed`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:51:44.160399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:44.167573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1242/2482] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:51:51.668000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:53.196325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:53.196912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:53.196912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:53.216765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:51:53.218571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:53.292123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1241/2483] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:51:53.459654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:51:53.461237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:51:53.516545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:51:53.537420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd558f4-5d3d-4220-8775-c8a446c8a213 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:51:53.542879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627966-9d60-4139-8090-9423e6ce5d0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:51:53.543523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627966-9d60-4139-8090-9423e6ce5d0a service swift] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:51:53.543803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627966-9d60-4139-8090-9423e6ce5d0a service swift] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:51:53.544072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627966-9d60-4139-8090-9423e6ce5d0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:51:53.552541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627966-9d60-4139-8090-9423e6ce5d0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:51:53.554144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627966-9d60-4139-8090-9423e6ce5d0a service swift] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:51:53.562983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1243/2484] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:51:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:52:03.282774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=25 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:03.725838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=10 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:10.244443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:10.245968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:10.258404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb7d3d-2d81-44e3-b91a-22feb16bfab9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:10.262001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:10.262601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:10.262880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:10.263154 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:10.264947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:10.274504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:10.275921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:10.284050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1242/2485] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:13.671508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:13.672099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:13.672099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:13.691538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:13.692721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:13.701631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1244/2486] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:13.713931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:13.727416 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17043aa-9531-4360-a5a6-5e0143d0884e service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:13.732975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:13.733579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:13.733857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:13.734125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:13.742965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:13.744748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:13.753745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1243/2487] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:52:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:52:17.655405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:17.665963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddf0e7-febd-4ae0-9ebd-71c6f4e2b40f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:17.669692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:17.670298 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:17.670576 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:17.670844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:17.673294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:17.685914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:17.693936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:17.698947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1245/2488] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:22.282481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:22.285383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:22.358273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9d19-6d1c-4814-847b-e534249d8029 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:22.363279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:22.364511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:22.364511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:22.364511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:22.366143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:22.377613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:22.379180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:22.389788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1244/2489] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:22.396301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:22.402839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:22.413061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747547-1a46-491d-8ec8-1abc2b4cd89e service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:22.417978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:22.418849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:22.419126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:22.419577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] PATH_INFO: `/v3/users/8eb5509df78d45acbebc332a889251dc` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:22.420728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb5509df78d45acbebc332a889251dc)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:22.422724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:22.601888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb5509df78d45acbebc332a889251dc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:22.662611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb5509df78d45acbebc332a889251dc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:22.670893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb5509df78d45acbebc332a889251dc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:22.671534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb5509df78d45acbebc332a889251dc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:22.672167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-02b6ca96-a1de-4e94-a994-b66ff6e98931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb5509df78d45acbebc332a889251dc'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:22.689933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1246/2490] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:52:22 2022] DELETE /identity/v3/users/8eb5509df78d45acbebc332a889251dc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:22.933001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:22.944735 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd73e6f-aafd-499d-883a-12701ca59006 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.002449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.003075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.003352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.003614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] PATH_INFO: `/v3/projects/e03d56845a594ecf9c320b2d70a633ca` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.004753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03d56845a594ecf9c320b2d70a633ca)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.006777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.101627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03d56845a594ecf9c320b2d70a633ca', 'request_id': 'req-3850edb1-b72b-4c6e-a217-6153ba327429'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:23.157593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03d56845a594ecf9c320b2d70a633ca', 'request_id': 'req-3850edb1-b72b-4c6e-a217-6153ba327429'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:23.163691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03d56845a594ecf9c320b2d70a633ca', 'request_id': 'req-3850edb1-b72b-4c6e-a217-6153ba327429'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:23.167596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:23.175842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] The token cache is being invalidate because project e03d56845a594ecf9c320b2d70a633ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:52:23.176076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:23.176823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3850edb1-b72b-4c6e-a217-6153ba327429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:23.178307 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1245/2491] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:52:22 2022] DELETE /identity/v3/projects/e03d56845a594ecf9c320b2d70a633ca => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:52:23.183253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.183519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.183793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.204177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:23.205375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:23.265563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1247/2492] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:23.273952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.274241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.274518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.295394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:23.296689 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:23.305497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1246/2493] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:23.309669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:23.319411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6a04e-a4c3-475e-874b-cc820b11e8ac None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.324190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.324785 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.325083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.325358 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.326001 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:52:23.326299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.327900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.339777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1248/2494] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:52:23.344650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:23.354202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cc9fa-d5a7-493c-9354-2646434d55a9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.358904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.360288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.360566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.360831 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.361474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.363382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.481060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1247/2495] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:52:23 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:23.485712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:23.496021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb768ae-2f68-426e-832e-45e27a5d00a5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.533999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:23.553037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.553637 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.553915 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.554289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.554843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.556858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.658078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1249/2496] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:52:23 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:23.662548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:23.673795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30f2c-55bf-4257-b059-48a856da55f8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.678478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.679081 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.679399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.679672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.680376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.681976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.692056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1248/2497] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:23.696063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:23.706732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b299ad9-5821-4654-896e-f5449c8c210c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.711385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424812c-dffc-4a31-abc2-f794903b5e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.715029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424812c-dffc-4a31-abc2-f794903b5e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.715029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424812c-dffc-4a31-abc2-f794903b5e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.715029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424812c-dffc-4a31-abc2-f794903b5e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.715029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424812c-dffc-4a31-abc2-f794903b5e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.715029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424812c-dffc-4a31-abc2-f794903b5e0a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.725153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1250/2498] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:23.729318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:23.739242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cfb39-dcc9-471e-a268-783715b2acb2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:23.743877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:23.744511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.744786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.745054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] PATH_INFO: `/v3/projects/18f2500a7d554360b088893254c5d12b/users/b8363d251a1042d4868081d36cfb9024/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.759446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f2500a7d554360b088893254c5d12b, user_id=b8363d251a1042d4868081d36cfb9024, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:23.762177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:23.832353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1249/2499] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:23 2022] PUT /identity/v3/projects/18f2500a7d554360b088893254c5d12b/users/b8363d251a1042d4868081d36cfb9024/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:23.842719 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:23.843004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:23.843280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:23.876173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] MFA Rules not processed for user `b8363d251a1042d4868081d36cfb9024`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:23.877397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:23.942849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1251/2500] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:23.947414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.011820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa4193-69bb-43e4-8828-f19b91a45e4c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.017357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.023501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.023501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.023501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.023501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.023501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.101324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1250/2501] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:52:23 2022] POST /identity/v3/projects => generated 403 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:24.105460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.116086 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce5474-0154-4793-84a2-0e3ca2495c65 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.172065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.172701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.172983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.173259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.173895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.175821 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.272697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1252/2502] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:52:24 2022] POST /identity/v3/users => generated 308 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:24.276594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.287089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b7c70-7698-433c-810a-5ce06ef60c14 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.291880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.292534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.292815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.293130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.293777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.295371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.306109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1251/2503] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:24.310241 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.320834 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc24c1-88de-44b6-9a77-bbba53bb609b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.325492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.326136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.326426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.326688 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] PATH_INFO: `/v3/projects/b1e0bf4055e145fbac55cdb94c82cf45/users/5afa1b936aa541c8b3e06bd7ea4c66b1/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.341701 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e0bf4055e145fbac55cdb94c82cf45, user_id=5afa1b936aa541c8b3e06bd7ea4c66b1, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.344683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.398144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1253/2504] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:24 2022] PUT /identity/v3/projects/b1e0bf4055e145fbac55cdb94c82cf45/users/5afa1b936aa541c8b3e06bd7ea4c66b1/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:24.402220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.412485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fca6d-6035-4432-854d-932838d60cba admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.469478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f10919-cb0f-4a2e-8705-fde9cd370f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.470091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f10919-cb0f-4a2e-8705-fde9cd370f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.470370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f10919-cb0f-4a2e-8705-fde9cd370f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.470641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f10919-cb0f-4a2e-8705-fde9cd370f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.471274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f10919-cb0f-4a2e-8705-fde9cd370f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.472914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f10919-cb0f-4a2e-8705-fde9cd370f41 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.478703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:24.483695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1252/2505] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:24.488527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.498804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5fe20-fa27-4535-a363-79bab2fc73f1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.504068 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.504751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.505030 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.505301 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] PATH_INFO: `/v3/projects/b1e0bf4055e145fbac55cdb94c82cf45/users/5afa1b936aa541c8b3e06bd7ea4c66b1/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.507632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e0bf4055e145fbac55cdb94c82cf45, user_id=5afa1b936aa541c8b3e06bd7ea4c66b1, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.510464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.517132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:24.521943 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1254/2506] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:24 2022] PUT /identity/v3/projects/b1e0bf4055e145fbac55cdb94c82cf45/users/5afa1b936aa541c8b3e06bd7ea4c66b1/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:24.533648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.533844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.534113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.563687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] MFA Rules not processed for user `5afa1b936aa541c8b3e06bd7ea4c66b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:24.565023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:24.636178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1253/2507] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:24.647540 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.649485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:24.708326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795477d5-d195-48af-9e1f-c67e4693f785 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.711984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb21389-291a-41df-9292-54ac02dab9e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.712684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb21389-291a-41df-9292-54ac02dab9e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.713631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb21389-291a-41df-9292-54ac02dab9e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.713631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb21389-291a-41df-9292-54ac02dab9e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.722798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb21389-291a-41df-9292-54ac02dab9e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.724315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb21389-291a-41df-9292-54ac02dab9e0 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:24.733539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1255/2508] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:24.876317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:24.938237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00e6ac-9dd2-47b8-b584-b25f6788c908 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:24.942760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:24.943382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:24.943656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:24.943946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] PATH_INFO: `/v3/users/b8363d251a1042d4868081d36cfb9024` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:24.945080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8363d251a1042d4868081d36cfb9024)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:24.946989 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:25.134162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8363d251a1042d4868081d36cfb9024'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:25.134820 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8363d251a1042d4868081d36cfb9024'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:25.135522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8363d251a1042d4868081d36cfb9024'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:25.199649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8363d251a1042d4868081d36cfb9024'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:25.207890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8363d251a1042d4868081d36cfb9024'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:25.225463 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1254/2509] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:52:24 2022] DELETE /identity/v3/users/b8363d251a1042d4868081d36cfb9024 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:25.229911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:25.230195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:25.230460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:25.249599 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:25.251419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:25.310563 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1256/2510] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:52:25.321048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:25.322581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:25.329065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:25.381934 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4ca0e-e992-4863-9a3f-2908e7b38f2f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:25.385774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:25.386386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:25.386664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:25.386991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:25.397344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:25.398905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:25.410149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1255/2511] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:25.864679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:25.866221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:25.878482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812c51a-0729-441c-a8eb-65c166b31f33 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:25.886103 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:25.886739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:25.887013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:25.887279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] PATH_INFO: `/v3/projects/18f2500a7d554360b088893254c5d12b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:25.888456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f2500a7d554360b088893254c5d12b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:25.890459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:26.005442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f2500a7d554360b088893254c5d12b', 'request_id': 'req-6382724c-a22f-498f-9719-8ae53563d9de'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.077043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f2500a7d554360b088893254c5d12b', 'request_id': 'req-6382724c-a22f-498f-9719-8ae53563d9de'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.083339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f2500a7d554360b088893254c5d12b', 'request_id': 'req-6382724c-a22f-498f-9719-8ae53563d9de'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.094379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] The token cache is being invalidate because project 18f2500a7d554360b088893254c5d12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:52:26.096239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.096239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.097520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1257/2512] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:52:25 2022] DELETE /identity/v3/projects/18f2500a7d554360b088893254c5d12b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:26.101661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:26.111242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edf475-dc34-44fe-a814-cb7ab2b68601 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:26.168133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:26.168880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:26.169183 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:26.169455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] PATH_INFO: `/v3/users/5afa1b936aa541c8b3e06bd7ea4c66b1` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:26.170567 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afa1b936aa541c8b3e06bd7ea4c66b1)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:26.172517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:26.373109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afa1b936aa541c8b3e06bd7ea4c66b1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.373755 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afa1b936aa541c8b3e06bd7ea4c66b1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.374301 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afa1b936aa541c8b3e06bd7ea4c66b1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.450861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afa1b936aa541c8b3e06bd7ea4c66b1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.458668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-be6e585f-3bba-4380-a35d-6a9c5b04e623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afa1b936aa541c8b3e06bd7ea4c66b1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:26.475908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1256/2513] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:52:26 2022] DELETE /identity/v3/users/5afa1b936aa541c8b3e06bd7ea4c66b1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:26.919255 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:26.920777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:26.936903 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382724c-a22f-498f-9719-8ae53563d9de admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:26.969504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:27.007471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:27.008112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:27.018648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:27.018648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] PATH_INFO: `/v3/projects/b1e0bf4055e145fbac55cdb94c82cf45` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:27.018648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e0bf4055e145fbac55cdb94c82cf45)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:27.018648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.141612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e0bf4055e145fbac55cdb94c82cf45', 'request_id': 'req-58c81e41-9763-4260-a863-718e576e618e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:28.208851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e0bf4055e145fbac55cdb94c82cf45', 'request_id': 'req-58c81e41-9763-4260-a863-718e576e618e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:28.215261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e0bf4055e145fbac55cdb94c82cf45', 'request_id': 'req-58c81e41-9763-4260-a863-718e576e618e'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:28.226766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] The token cache is being invalidate because project b1e0bf4055e145fbac55cdb94c82cf45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:52:28.227002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:28.227724 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-58c81e41-9763-4260-a863-718e576e618e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:28.229573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1258/2514] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:52:26 2022] DELETE /identity/v3/projects/b1e0bf4055e145fbac55cdb94c82cf45 => generated 0 bytes in 1311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:28.236265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.236265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.236265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.257977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:28.259199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:28.281943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:28.323440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1257/2515] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:28.331444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.331748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.332017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.353264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:28.354523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:28.364350 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1259/2516] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:28.368512 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:28.379480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baad77b-cb89-4ea3-b76f-2720f6c3f315 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:28.387773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:28.388451 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.393288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.393288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.393288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:52:28.393288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:28.393288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.403747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1258/2517] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:52:28.408221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:28.419870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd973ac-5161-4ca7-931e-07e7b368529f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:28.426863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:28.427459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.427743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.428014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.428678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:28.430603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.515425 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1260/2518] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:52:28 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:28.519546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:28.529870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc760e-f6ab-46a4-9109-3b888d8bcb65 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:28.584847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:28.585441 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.585719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.585986 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.586610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:28.588548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.707089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1259/2519] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:52:28 2022] POST /identity/v3/users => generated 281 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:52:28.715195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:28.730335 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bc86d-b5ee-4ad0-9b10-0d9f603e5720 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:28.735344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:28.737633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.737633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.737633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.737999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:28.739489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.750062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1261/2520] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:28.754396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:28.771470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481cb79-dc44-492f-bbe8-e1d1033eff12 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:28.773675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca266a-8d95-4b9a-9007-6d604e4260d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:28.774283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca266a-8d95-4b9a-9007-6d604e4260d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.774564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca266a-8d95-4b9a-9007-6d604e4260d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.774838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca266a-8d95-4b9a-9007-6d604e4260d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.775482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca266a-8d95-4b9a-9007-6d604e4260d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:28.777120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca266a-8d95-4b9a-9007-6d604e4260d6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.788347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1260/2521] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:28.792709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:28.803577 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c0953-40f8-47f8-8312-c572ac166a9f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:28.808188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:28.808869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.809147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.809417 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] PATH_INFO: `/v3/projects/05dc7d112f5a4ec1944ae895c31f7222/users/eaa16678c1514992a5f0fb2d4c2ba2fe/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.823775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05dc7d112f5a4ec1944ae895c31f7222, user_id=eaa16678c1514992a5f0fb2d4c2ba2fe, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:28.826463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:28.925126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1262/2522] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:28 2022] PUT /identity/v3/projects/05dc7d112f5a4ec1944ae895c31f7222/users/eaa16678c1514992a5f0fb2d4c2ba2fe/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:28.936779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:28.937073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:28.937374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:28.965354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] MFA Rules not processed for user `eaa16678c1514992a5f0fb2d4c2ba2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:28.966611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:29.033091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1261/2523] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:29.037705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.096105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:29.106518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9f8d6-bb5c-42f8-8b9b-37bb95e75bd6 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.111331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.111926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.112235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.112510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.113141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.115062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.215409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1263/2524] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:52:29 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:29.219352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.230747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb397-2a5c-48d2-9d7a-a645eb2bc327 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.249526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:29.289524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.290061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.290343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.290617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.291251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.293317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.383550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1262/2525] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:52:29 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:29.387507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.399168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428c76d-f2f5-498f-837a-d72181e4ff15 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.403989 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.404608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.404894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.405166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.405806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.407407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.417566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1264/2526] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:29.421708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.431783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e250126-02db-48b9-b909-2022360caf53 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.436384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.437039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.437318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.437593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] PATH_INFO: `/v3/projects/cd3aebe7c998493eaadec1245fb7e969/users/088923beea744ca0ad3db36eb0c8c1d5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.453221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3aebe7c998493eaadec1245fb7e969, user_id=088923beea744ca0ad3db36eb0c8c1d5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.456538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.534731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1263/2527] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:29 2022] PUT /identity/v3/projects/cd3aebe7c998493eaadec1245fb7e969/users/088923beea744ca0ad3db36eb0c8c1d5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:29.538788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.549570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3835b1-5eb3-42de-90a8-399d48e9217a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.606520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01eb89b-567d-421a-b754-4cab365d3a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.607124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eb89b-567d-421a-b754-4cab365d3a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.607406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eb89b-567d-421a-b754-4cab365d3a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.607681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eb89b-567d-421a-b754-4cab365d3a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.608422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01eb89b-567d-421a-b754-4cab365d3a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.610078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01eb89b-567d-421a-b754-4cab365d3a8e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.620009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1265/2528] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:29.624365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.635129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac34c3-bcd4-4f60-be1b-6a0ba51c377f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.639764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.640436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.640708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.640973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] PATH_INFO: `/v3/projects/cd3aebe7c998493eaadec1245fb7e969/users/088923beea744ca0ad3db36eb0c8c1d5/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.643448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3aebe7c998493eaadec1245fb7e969, user_id=088923beea744ca0ad3db36eb0c8c1d5, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.646369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.656353 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1264/2529] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:29 2022] PUT /identity/v3/projects/cd3aebe7c998493eaadec1245fb7e969/users/088923beea744ca0ad3db36eb0c8c1d5/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:29.666341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ca1d6-eb12-4368-acea-e50e37756f49 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.666628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ca1d6-eb12-4368-acea-e50e37756f49 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.666897 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ca1d6-eb12-4368-acea-e50e37756f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.696761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-624ca1d6-eb12-4368-acea-e50e37756f49 None None] MFA Rules not processed for user `088923beea744ca0ad3db36eb0c8c1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:29.697985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-624ca1d6-eb12-4368-acea-e50e37756f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:29.765451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1266/2530] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:29.777392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:29.779082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:29.836630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833eda4-8bc1-4faa-8bc4-8f3a1d8aba8b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:29.840176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:29.840774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.841046 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.841312 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.850668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:29.852080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:29.861382 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1265/2531] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:29.920069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:29.920418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:29.920698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:29.939921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:29.941156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:29.987757 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:29.999402 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1267/2532] 10.222.0.57 () {58 vars in 967 bytes} [Wed Sep 7 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:52:30.013448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:30.014649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:30.085054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c31be-3e90-4481-9f08-a736a7a9ea3e service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:30.098365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d8023-0f8f-493d-8676-c6444f4c7f8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:30.098365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d8023-0f8f-493d-8676-c6444f4c7f8d service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:30.098365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d8023-0f8f-493d-8676-c6444f4c7f8d service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:30.098365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d8023-0f8f-493d-8676-c6444f4c7f8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:30.099727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d8023-0f8f-493d-8676-c6444f4c7f8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:30.101321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d8023-0f8f-493d-8676-c6444f4c7f8d service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:30.109927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1266/2533] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:52:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:52:30.766126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:30.832949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d59b-4cc1-46ef-b4b3-f51ae3c9f0ea None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:30.837523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:30.838140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:30.838415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:30.838680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] PATH_INFO: `/v3/users/eaa16678c1514992a5f0fb2d4c2ba2fe` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:30.839827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa16678c1514992a5f0fb2d4c2ba2fe)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:30.841832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:31.022681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa16678c1514992a5f0fb2d4c2ba2fe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.082933 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa16678c1514992a5f0fb2d4c2ba2fe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.090516 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa16678c1514992a5f0fb2d4c2ba2fe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.091194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa16678c1514992a5f0fb2d4c2ba2fe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.091858 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa16678c1514992a5f0fb2d4c2ba2fe'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.109793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1268/2534] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:52:30 2022] DELETE /identity/v3/users/eaa16678c1514992a5f0fb2d4c2ba2fe => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:31.114888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:31.115392 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:31.115506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:31.128583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:31.135244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:31.136530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:31.198413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1267/2535] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:31.209113 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:31.210666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:31.269099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa1398-2e09-4a79-a373-bcc91018e670 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:31.273160 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:31.273748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:31.274022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:31.274290 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:31.284063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:31.285536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:31.295291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1269/2536] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:31.719208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:31.720879 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:31.733588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186e34f-cf0e-42ef-8991-eae37a5315f6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:31.739185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:31.739955 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:31.740367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:31.740929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] PATH_INFO: `/v3/projects/05dc7d112f5a4ec1944ae895c31f7222` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:31.742170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05dc7d112f5a4ec1944ae895c31f7222)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:31.745117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:31.871805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05dc7d112f5a4ec1944ae895c31f7222', 'request_id': 'req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.928538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05dc7d112f5a4ec1944ae895c31f7222', 'request_id': 'req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.934528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05dc7d112f5a4ec1944ae895c31f7222', 'request_id': 'req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.946455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] The token cache is being invalidate because project 05dc7d112f5a4ec1944ae895c31f7222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:52:31.946745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.947383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:31.948914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1268/2537] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:52:31 2022] DELETE /identity/v3/projects/05dc7d112f5a4ec1944ae895c31f7222 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:31.953081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:31.962977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6ac56-0111-4d6a-8bf1-1225bab1513f service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:31.975204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:32.021317 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:32.021968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:32.022249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:32.022521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] PATH_INFO: `/v3/users/088923beea744ca0ad3db36eb0c8c1d5` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:32.023635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088923beea744ca0ad3db36eb0c8c1d5)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:32.025616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:32.218211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088923beea744ca0ad3db36eb0c8c1d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:32.305726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088923beea744ca0ad3db36eb0c8c1d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:32.312825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088923beea744ca0ad3db36eb0c8c1d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:32.313446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088923beea744ca0ad3db36eb0c8c1d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:32.314052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0242fdc1-eca2-4b28-8d53-f2c0dc2bb052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088923beea744ca0ad3db36eb0c8c1d5'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:32.331645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1270/2538] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:52:31 2022] DELETE /identity/v3/users/088923beea744ca0ad3db36eb0c8c1d5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:32.744854 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:32.746379 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:32.761185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3dae-dd2e-42f6-8fb1-fedb993a6a00 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:32.824476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:32.825304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:32.825628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:32.825904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] PATH_INFO: `/v3/projects/cd3aebe7c998493eaadec1245fb7e969` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:32.827144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3aebe7c998493eaadec1245fb7e969)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:32.830517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:32.920071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:32.932241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3aebe7c998493eaadec1245fb7e969', 'request_id': 'req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:33.028222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3aebe7c998493eaadec1245fb7e969', 'request_id': 'req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:33.034548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3aebe7c998493eaadec1245fb7e969', 'request_id': 'req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:33.045916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] The token cache is being invalidate because project cd3aebe7c998493eaadec1245fb7e969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:52:33.046204 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:33.046853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a7c8fe24-9dfa-4f15-abb9-ebc1ae619846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:52:33.048369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1269/2539] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:52:32 2022] DELETE /identity/v3/projects/cd3aebe7c998493eaadec1245fb7e969 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:33.053718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.054188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.054672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.076899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:33.078979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:33.140712 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1271/2540] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:33.149437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.149723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.149996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.169647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:33.170859 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:33.179329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1270/2541] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:52:33.183573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.195644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea3b74-abaf-4434-8c04-c8c3ae05925d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.200398 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.201002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.201281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.201551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.202229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:52:33.202502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.204742 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:33.217824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1272/2542] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:52:33.224598 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.233865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5235f3-fe61-4885-9085-2f083f0722a9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.238627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.239240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.239759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.240315 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.241381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.243419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:33.324708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1271/2543] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:52:33 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:33.328789 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.338902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85787b98-c9e7-4c52-8138-58f8d0860bc5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.358461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:33.395991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.396624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.396929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.397259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.397894 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.399817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:33.485353 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1273/2544] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:52:33 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:33.489417 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.499663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d723d2-6d89-4e42-8247-8ea8fb2b32b7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.504372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.504965 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.505240 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.505504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.506134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.508918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:33.519340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1272/2545] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:33.523662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.534131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b0f4-c95e-4d28-8ff1-73e31b46fbf4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.538826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95d36-7842-44ee-bb74-b8659bb8d982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.540522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95d36-7842-44ee-bb74-b8659bb8d982 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.540522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95d36-7842-44ee-bb74-b8659bb8d982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.540522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95d36-7842-44ee-bb74-b8659bb8d982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.540850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95d36-7842-44ee-bb74-b8659bb8d982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.542221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95d36-7842-44ee-bb74-b8659bb8d982 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:33.554138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1274/2546] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:33.558198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.568043 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ed8bf-069b-4a6a-9ab6-486ed3dbb027 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.572653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.573294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.573572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.573845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] PATH_INFO: `/v3/projects/cf7ae86ff0f64209951467219d70a380/users/26cc35920ea847f4b5664ffe4d58f3fa/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.588688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7ae86ff0f64209951467219d70a380, user_id=26cc35920ea847f4b5664ffe4d58f3fa, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.591266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:33.702443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1273/2547] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:33 2022] PUT /identity/v3/projects/cf7ae86ff0f64209951467219d70a380/users/26cc35920ea847f4b5664ffe4d58f3fa/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:33.714111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.714391 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.714663 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.742249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] MFA Rules not processed for user `26cc35920ea847f4b5664ffe4d58f3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:33.743451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:33.807410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1275/2548] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:33.812070 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:33.879145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdd910-c447-4fdf-b849-1feb778be5c4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:33.884028 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:33.884664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:33.884940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:33.885208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:33.885861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:33.887786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.001296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1274/2549] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:52:33 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:34.005337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:34.015890 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cafc7d-9fc7-4eb6-bfbe-89f3eb074f7d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:34.073242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:34.073862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.074106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.074394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.075007 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:34.076952 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.182157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1276/2550] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:52:34 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:52:34.186125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:34.197110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11782c1e-b2ab-4422-b48f-78440e52866f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:34.201772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:34.202375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.202652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.202921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.203559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:34.205210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.219841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1275/2551] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:34.222729 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:34.233004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff23a5-5519-44fe-a7b6-2069b4f60602 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:34.238171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:34.238824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.239101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.239396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] PATH_INFO: `/v3/projects/1d2c7df93a2e42f1bc8434b45ff0f260/users/32462d4a77284806ac1620822eb95e65/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.254679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2c7df93a2e42f1bc8434b45ff0f260, user_id=32462d4a77284806ac1620822eb95e65, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:34.257390 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.260447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:34.334459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1277/2552] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:34 2022] PUT /identity/v3/projects/1d2c7df93a2e42f1bc8434b45ff0f260/users/32462d4a77284806ac1620822eb95e65/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:34.338475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:34.348431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ef382-f29d-49e6-b1c5-c13cfbe0feff admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:34.379458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:34.408970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b00b-f7e4-4070-8aab-60e3f203202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:34.409571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b00b-f7e4-4070-8aab-60e3f203202c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.409846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b00b-f7e4-4070-8aab-60e3f203202c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.410110 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b00b-f7e4-4070-8aab-60e3f203202c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.410733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8b00b-f7e4-4070-8aab-60e3f203202c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:34.412373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8b00b-f7e4-4070-8aab-60e3f203202c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.421919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1276/2553] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:52:34.426629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:34.446137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8abe-f898-48ea-a2fd-08cd0a1add16 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:34.446137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:34.446137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.446137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.446640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] PATH_INFO: `/v3/projects/1d2c7df93a2e42f1bc8434b45ff0f260/users/32462d4a77284806ac1620822eb95e65/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.447454 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2c7df93a2e42f1bc8434b45ff0f260, user_id=32462d4a77284806ac1620822eb95e65, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:34.450221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.462029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1278/2554] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:52:34 2022] PUT /identity/v3/projects/1d2c7df93a2e42f1bc8434b45ff0f260/users/32462d4a77284806ac1620822eb95e65/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:52:34.472287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.474760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.474760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.501492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] MFA Rules not processed for user `32462d4a77284806ac1620822eb95e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:34.502725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:34.573074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1277/2555] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:52:34.584803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:34.586650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:34.646857 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4062b-8688-4c23-919a-3a6720859db3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:34.650721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9935a25-aec0-47d3-9354-042c6ef6b5f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:34.651340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9935a25-aec0-47d3-9354-042c6ef6b5f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:34.651619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9935a25-aec0-47d3-9354-042c6ef6b5f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:34.651891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9935a25-aec0-47d3-9354-042c6ef6b5f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:34.710218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9935a25-aec0-47d3-9354-042c6ef6b5f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:34.711601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9935a25-aec0-47d3-9354-042c6ef6b5f9 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:34.720567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1279/2556] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:40.426492 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:52:40.427890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:40.440274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b4a16-fd23-4ffc-a5f2-3fc10b5f31d0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:52:40.444049 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:52:40.444684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:40.444964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:40.445238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:40.455900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:52:40.457435 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:52:40.469060 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1278/2557] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:52:41.432848 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8122-3a35-4970-9588-2499e7d68e58 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:52:41.433458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8122-3a35-4970-9588-2499e7d68e58 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:52:41.433458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8122-3a35-4970-9588-2499e7d68e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:52:41.452968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d68d8122-3a35-4970-9588-2499e7d68e58 None None] MFA Rules not processed for user `2a6a637072b04667a3f21da9c57325c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:52:41.454339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d68d8122-3a35-4970-9588-2499e7d68e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:52:41.511582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1280/2558] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:52:50.463178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=55 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:52:51.505090 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:07.718673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:07.782550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e803f-6b1a-457b-8bc7-104a9c645255 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:07.787625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:07.788310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:07.788589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:07.788863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] PATH_INFO: `/v3/users/26cc35920ea847f4b5664ffe4d58f3fa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:07.790051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cc35920ea847f4b5664ffe4d58f3fa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:07.791941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:07.964653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cc35920ea847f4b5664ffe4d58f3fa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:07.965434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cc35920ea847f4b5664ffe4d58f3fa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:07.966068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cc35920ea847f4b5664ffe4d58f3fa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:08.043142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cc35920ea847f4b5664ffe4d58f3fa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:08.051189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cc35920ea847f4b5664ffe4d58f3fa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:08.075607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1279/2559] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:07 2022] DELETE /identity/v3/users/26cc35920ea847f4b5664ffe4d58f3fa => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:08.094342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62f479-098a-4db0-b750-884186102e86 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:08.094342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62f479-098a-4db0-b750-884186102e86 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:08.094836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62f479-098a-4db0-b750-884186102e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:08.114330 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9d62f479-098a-4db0-b750-884186102e86 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:08.116586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9d62f479-098a-4db0-b750-884186102e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:08.183739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1281/2560] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:08.193949 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:08.199371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:08.253363 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c1533-fecd-466d-a0d1-eb0f6a7fd6b6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:08.257256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:08.257928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:08.258211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:08.258484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:08.267845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:08.269317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:08.279455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1280/2561] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:08.858291 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62f479-098a-4db0-b750-884186102e86 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:08.860644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9d62f479-098a-4db0-b750-884186102e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:08.874805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62f479-098a-4db0-b750-884186102e86 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:08.881399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:08.882236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:08.884496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:08.884496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] PATH_INFO: `/v3/projects/cf7ae86ff0f64209951467219d70a380` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:08.884496 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7ae86ff0f64209951467219d70a380)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:08.886063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:09.024012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7ae86ff0f64209951467219d70a380', 'request_id': 'req-11d32f86-a94e-4a14-aae6-2e6becb20d56'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.117646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7ae86ff0f64209951467219d70a380', 'request_id': 'req-11d32f86-a94e-4a14-aae6-2e6becb20d56'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.123917 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7ae86ff0f64209951467219d70a380', 'request_id': 'req-11d32f86-a94e-4a14-aae6-2e6becb20d56'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.136279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] The token cache is being invalidate because project cf7ae86ff0f64209951467219d70a380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:09.136760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.137409 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.138919 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1282/2562] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:08 2022] DELETE /identity/v3/projects/cf7ae86ff0f64209951467219d70a380 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:09.143008 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:09.154231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f7f74-5f8a-4baa-9316-1f551f239966 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:09.173157 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:09.214027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:09.214671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:09.214947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:09.215216 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] PATH_INFO: `/v3/users/32462d4a77284806ac1620822eb95e65` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:09.216444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32462d4a77284806ac1620822eb95e65)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:09.218380 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:09.405571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32462d4a77284806ac1620822eb95e65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.406234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32462d4a77284806ac1620822eb95e65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.406867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32462d4a77284806ac1620822eb95e65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.482802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32462d4a77284806ac1620822eb95e65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.490891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6694246e-524f-4a2d-8438-2b014f3b9812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32462d4a77284806ac1620822eb95e65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:09.509708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1281/2563] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:09 2022] DELETE /identity/v3/users/32462d4a77284806ac1620822eb95e65 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:09.912548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:09.912548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:09.933074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32f86-a94e-4a14-aae6-2e6becb20d56 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:09.997964 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:09.998680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:09.999017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:09.999352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] PATH_INFO: `/v3/projects/1d2c7df93a2e42f1bc8434b45ff0f260` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.000559 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2c7df93a2e42f1bc8434b45ff0f260)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.002550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.134237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:10.154166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2c7df93a2e42f1bc8434b45ff0f260', 'request_id': 'req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:10.222419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2c7df93a2e42f1bc8434b45ff0f260', 'request_id': 'req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:10.229872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2c7df93a2e42f1bc8434b45ff0f260', 'request_id': 'req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:10.246485 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] The token cache is being invalidate because project 1d2c7df93a2e42f1bc8434b45ff0f260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:10.246784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:10.247608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-c22b4c0b-76f0-4018-b921-5fb9bd4c7eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:10.250418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1283/2564] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:09 2022] DELETE /identity/v3/projects/1d2c7df93a2e42f1bc8434b45ff0f260 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:10.254636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.255136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.255245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.276778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:10.278147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:10.360306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1282/2565] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:10.367489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.367775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.368072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.390309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:10.394539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:10.406826 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1284/2566] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:10.407983 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:10.418441 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20082f9-fa17-43fe-bec0-c62e83fa0b58 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:10.424510 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:10.425124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.425399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.425670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.426323 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:10.426620 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.428304 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.441637 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1283/2567] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:10.446188 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:10.457658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2f9dc-db65-4e74-b454-0a2f007c5693 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:10.470244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:10.470934 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.471217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.471509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.472183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.475780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.580885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1285/2568] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:10 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:10.582266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:10.595852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd2733-c67e-488e-83f2-53cee0ff4d47 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:10.642927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:10.677445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:10.678579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.678864 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.679152 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.679918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.681938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.783401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1284/2569] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:10 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:10.783702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:10.794258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e67eb0-2b1d-4a08-8e05-fe883302bf71 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:10.799134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:10.799746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.800025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.800320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.800960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.802633 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.812864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1286/2570] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:10.816872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:10.828480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be594a21-bd33-4a36-8140-2a26a0e65caa admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:10.834273 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43e9e6-ab20-4292-92b0-d09d425efc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:10.834932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43e9e6-ab20-4292-92b0-d09d425efc6b admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.835264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43e9e6-ab20-4292-92b0-d09d425efc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.835668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43e9e6-ab20-4292-92b0-d09d425efc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.836432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43e9e6-ab20-4292-92b0-d09d425efc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.843902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43e9e6-ab20-4292-92b0-d09d425efc6b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.848984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1285/2571] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:10.853819 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:10.864181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bbb39-7ade-4cbc-9918-20ddd94d342b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:10.869260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:10.870013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.870406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.870695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] PATH_INFO: `/v3/projects/bc6d03d00eae471990af7896c84961ad/users/aca8d1ca0399425294a96b2a8fc33c66/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.885722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6d03d00eae471990af7896c84961ad, user_id=aca8d1ca0399425294a96b2a8fc33c66, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:10.888457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:10.952451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1287/2572] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:10 2022] PUT /identity/v3/projects/bc6d03d00eae471990af7896c84961ad/users/aca8d1ca0399425294a96b2a8fc33c66/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:10.963491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:10.963779 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:10.964111 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:10.994846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] MFA Rules not processed for user `aca8d1ca0399425294a96b2a8fc33c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:10.996086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:11.063878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1286/2573] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:11.075535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:11.077220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:11.138273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d28c32-0662-481f-867c-8badb35d2203 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:11.142093 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:11.142823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:11.143245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:11.143582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:11.154185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:11.155635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:11.169281 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1288/2574] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:18.947713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:18.949138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:19.010270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76be612-0ded-4fd4-950c-e64ee7e9a016 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:19.014234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:19.014858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:19.015166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:19.015440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:19.017390 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:19.072574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:19.087254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:19.088726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:19.098813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1287/2575] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:21.162544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:22.891839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:22.893363 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:22.908206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90043ce-a2ec-4dfc-9d61-b1b2a60e2bfd service glance] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:22.913082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:22.913741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:22.914015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:22.914284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] PATH_INFO: `/v3/users/eeab924bf7a646779b48f57f14710e05` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:22.915414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeab924bf7a646779b48f57f14710e05)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:22.917357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:23.070850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeab924bf7a646779b48f57f14710e05'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.148228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeab924bf7a646779b48f57f14710e05'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.156851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeab924bf7a646779b48f57f14710e05'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.157515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeab924bf7a646779b48f57f14710e05'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.158124 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5cce8547-7629-48d3-8b4f-64aa40d1ad4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeab924bf7a646779b48f57f14710e05'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.176755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1289/2576] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:22 2022] DELETE /identity/v3/users/eeab924bf7a646779b48f57f14710e05 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:23.404674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:23.415997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019e78-c8c7-4aef-acae-464341773e60 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:23.602789 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:23.603425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:23.603703 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:23.603966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] PATH_INFO: `/v3/projects/fa5b6b66febc4070a509feee0b0838f6` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:23.605107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5b6b66febc4070a509feee0b0838f6)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:23.606973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:23.714088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5b6b66febc4070a509feee0b0838f6', 'request_id': 'req-0f349971-d74a-4304-af4d-654cd22b542b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.793364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5b6b66febc4070a509feee0b0838f6', 'request_id': 'req-0f349971-d74a-4304-af4d-654cd22b542b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.799497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5b6b66febc4070a509feee0b0838f6', 'request_id': 'req-0f349971-d74a-4304-af4d-654cd22b542b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.810538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] The token cache is being invalidate because project fa5b6b66febc4070a509feee0b0838f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:23.810856 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.811491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0f349971-d74a-4304-af4d-654cd22b542b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:23.813019 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1288/2577] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:23 2022] DELETE /identity/v3/projects/fa5b6b66febc4070a509feee0b0838f6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:23.818161 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:23.818517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:23.818791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:23.839146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:23.840368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:23.905584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1290/2578] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:23.913630 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:23.913914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:23.914187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:23.933552 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:23.934750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:23.942576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1289/2579] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:23.947331 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:23.957063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3c36b-0f77-402a-9c42-7c2f78250e1a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:23.962259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:23.962848 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:23.963125 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:23.963394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:23.964038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:23.964371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:23.965980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:23.978114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1291/2580] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:23.982388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:23.991882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc604189-a6ae-465f-ad2e-065b9c141ad4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:23.996545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:23.997136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:23.997410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:23.997679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:23.998300 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:24.000198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:24.086433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1290/2581] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:23 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:24.090389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:24.100376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd89ce-c979-4dc3-a0bb-07a67cf22f16 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:24.156134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:24.156767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:24.157049 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:24.157320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:24.157956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:24.159918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:24.167795 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:24.248429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1292/2582] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:24 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:24.252774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:24.263757 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46dbcd-ffbc-463b-95ce-057d0529a186 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:24.268801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:24.269408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:24.269684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:24.269951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:24.270582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:24.272605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:24.282701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1291/2583] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:24.286765 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:24.296799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374dccd-28be-438b-8e65-2104b04f3a6c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:24.301466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ec63e-54f7-46d3-97a8-7bae6d9d4d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:24.302063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ec63e-54f7-46d3-97a8-7bae6d9d4d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:24.302828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ec63e-54f7-46d3-97a8-7bae6d9d4d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:24.303099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ec63e-54f7-46d3-97a8-7bae6d9d4d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:24.303733 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ec63e-54f7-46d3-97a8-7bae6d9d4d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:24.305386 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ec63e-54f7-46d3-97a8-7bae6d9d4d5b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:24.315872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1293/2584] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:24.320482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:24.330298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f7278-df3d-4531-88f7-d211d59688c2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:24.334915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:24.335551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:24.335827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:24.336097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] PATH_INFO: `/v3/projects/9193c2d554e649d280251ad312e94ead/users/2084397f6d9a4416a53c225a13e025e6/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:24.350431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9193c2d554e649d280251ad312e94ead, user_id=2084397f6d9a4416a53c225a13e025e6, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:24.353059 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:24.415998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1292/2585] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:24 2022] PUT /identity/v3/projects/9193c2d554e649d280251ad312e94ead/users/2084397f6d9a4416a53c225a13e025e6/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:24.427678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:24.428137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:24.428262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:24.457189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] MFA Rules not processed for user `2084397f6d9a4416a53c225a13e025e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:24.458407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:24.523854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1294/2586] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:24.535092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:24.536814 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:24.551124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:24.615847 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ee155-4396-441f-8f23-7e46f5d1d648 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:24.622329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b2409-b473-4711-b519-9de47d0a627e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:24.622939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b2409-b473-4711-b519-9de47d0a627e service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:24.623211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b2409-b473-4711-b519-9de47d0a627e service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:24.623476 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b2409-b473-4711-b519-9de47d0a627e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:24.631839 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b2409-b473-4711-b519-9de47d0a627e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:24.633430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b2409-b473-4711-b519-9de47d0a627e service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:24.642067 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1293/2587] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:30.459173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:30.460672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:30.526149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110db8-ff0a-4338-a537-6e7d2759ee7a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:30.530921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:30.531546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:30.531912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:30.532215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] PATH_INFO: `/v3/users/aca8d1ca0399425294a96b2a8fc33c66` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:30.533332 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca8d1ca0399425294a96b2a8fc33c66)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:30.535242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:30.713157 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca8d1ca0399425294a96b2a8fc33c66'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:30.785731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca8d1ca0399425294a96b2a8fc33c66'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:30.793628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca8d1ca0399425294a96b2a8fc33c66'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:30.794275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca8d1ca0399425294a96b2a8fc33c66'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:30.794874 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca8d1ca0399425294a96b2a8fc33c66'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:30.800048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:30.813126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1295/2588] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:30 2022] DELETE /identity/v3/users/aca8d1ca0399425294a96b2a8fc33c66 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:30.818161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:30.818650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:30.818650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:30.837021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:30.838282 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:30.897144 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1294/2589] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:30.907045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:30.908491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:30.967422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc931f-f5b2-44c9-a877-30486fab667e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:30.971096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485e6b8-481e-4330-9738-e5f0e0fed6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:30.971703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485e6b8-481e-4330-9738-e5f0e0fed6ce service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:30.971983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485e6b8-481e-4330-9738-e5f0e0fed6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:30.972288 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485e6b8-481e-4330-9738-e5f0e0fed6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:30.981704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485e6b8-481e-4330-9738-e5f0e0fed6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:30.983167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485e6b8-481e-4330-9738-e5f0e0fed6ce service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:30.993133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1296/2590] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:31.412845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:31.415015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:31.426455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1ed8-2954-49b3-b516-0da4d0fce45f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:31.431751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:31.432423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:31.432702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:31.432973 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] PATH_INFO: `/v3/projects/bc6d03d00eae471990af7896c84961ad` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:31.434712 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6d03d00eae471990af7896c84961ad)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:31.436002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:31.565641 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6d03d00eae471990af7896c84961ad', 'request_id': 'req-55dcc6a4-e0b5-478f-9471-72c4d630f681'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:31.638971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6d03d00eae471990af7896c84961ad', 'request_id': 'req-55dcc6a4-e0b5-478f-9471-72c4d630f681'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:31.644967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6d03d00eae471990af7896c84961ad', 'request_id': 'req-55dcc6a4-e0b5-478f-9471-72c4d630f681'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:31.656763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] The token cache is being invalidate because project bc6d03d00eae471990af7896c84961ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:31.657077 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:31.657741 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-55dcc6a4-e0b5-478f-9471-72c4d630f681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:31.659257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1295/2591] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:31 2022] DELETE /identity/v3/projects/bc6d03d00eae471990af7896c84961ad => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:31.664690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:31.665165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:31.665271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:31.685643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:31.686860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:31.749849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1297/2592] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:31.757647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:31.757945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:31.758219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:31.779094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:31.780371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:31.788897 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1296/2593] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:31.793122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:31.802831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4cb3a-f155-4371-8592-3679ffc014f7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:31.807631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:31.808261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:31.808543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:31.808813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:31.809480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:31.809782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:31.811383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:31.823092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1298/2594] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:31.827470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:31.837228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5a08b-511a-4b6e-a860-6a49e80fe693 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:31.841810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:31.842396 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:31.842670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:31.842935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:31.843578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:31.845478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:31.861730 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:31.929426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1297/2595] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:31 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:31.933872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:31.943920 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8be2ec-c40c-43e4-ad84-bcb709564c88 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:32.001220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:32.001854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:32.002135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:32.002419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:32.003044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:32.005018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:32.121098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1299/2596] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:31 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:53:32.122245 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:32.133455 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ef6c1-463b-4f0f-8916-06ba223a2f79 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:32.138576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:32.139196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:32.139472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:32.139743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:32.140427 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:32.142021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:32.154299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1298/2597] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:32.159814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:32.170226 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d7778-eb81-49fe-a390-764d48374a50 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:32.174996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e804de9-2b86-4aec-9291-b9f545d78620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:32.175669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804de9-2b86-4aec-9291-b9f545d78620 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:32.176016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804de9-2b86-4aec-9291-b9f545d78620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:32.176383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804de9-2b86-4aec-9291-b9f545d78620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:32.177105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e804de9-2b86-4aec-9291-b9f545d78620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:32.178822 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e804de9-2b86-4aec-9291-b9f545d78620 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:32.180224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:32.190618 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1300/2598] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:32.195521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:32.205840 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eff286-dc80-427e-a0bd-c8fa9c8d5773 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:32.210991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:32.211656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:32.211930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:32.212224 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] PATH_INFO: `/v3/projects/7128afc4275e47dfbc5d8fb5d49ea58a/users/ff43bfa4484e4c1889d66eb9aab61374/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:32.226932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7128afc4275e47dfbc5d8fb5d49ea58a, user_id=ff43bfa4484e4c1889d66eb9aab61374, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:32.229608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:32.318914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1299/2599] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:32 2022] PUT /identity/v3/projects/7128afc4275e47dfbc5d8fb5d49ea58a/users/ff43bfa4484e4c1889d66eb9aab61374/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:32.329775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:32.330060 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:32.330912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:32.360248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] MFA Rules not processed for user `ff43bfa4484e4c1889d66eb9aab61374`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:32.361512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:32.427323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1301/2600] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:32.439064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:32.444760 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:32.499131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a946a6e-c86a-4d8d-8128-565648832248 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:32.503002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289a4d0-dfd1-4204-8af4-fa2f21bae9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:32.503663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a4d0-dfd1-4204-8af4-fa2f21bae9ed service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:32.503943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a4d0-dfd1-4204-8af4-fa2f21bae9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:32.504241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a4d0-dfd1-4204-8af4-fa2f21bae9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:32.513495 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289a4d0-dfd1-4204-8af4-fa2f21bae9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:32.514969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289a4d0-dfd1-4204-8af4-fa2f21bae9ed service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:32.524210 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1300/2601] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:33.793322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:33.793322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:33.851232 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50118747-f601-43cd-b049-eabcabc79ef6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:33.855963 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:33.856629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:33.856909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:33.857178 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] PATH_INFO: `/v3/users/2084397f6d9a4416a53c225a13e025e6` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:33.858295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2084397f6d9a4416a53c225a13e025e6)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:33.860308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:34.065797 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2084397f6d9a4416a53c225a13e025e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:34.153399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2084397f6d9a4416a53c225a13e025e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:34.160835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2084397f6d9a4416a53c225a13e025e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:34.161476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2084397f6d9a4416a53c225a13e025e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:34.162082 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2084397f6d9a4416a53c225a13e025e6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:34.179836 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1302/2602] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:33 2022] DELETE /identity/v3/users/2084397f6d9a4416a53c225a13e025e6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:34.185527 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:34.186024 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:34.186135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:34.206086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:34.207343 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:34.268418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1301/2603] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:34.279352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:34.280820 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:34.287541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:34.338925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00134c2e-a0c1-418a-becb-11f3feaac671 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:34.343175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8fc37-f9a2-4f1d-bbb4-8f2d3111e043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:34.343769 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8fc37-f9a2-4f1d-bbb4-8f2d3111e043 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:34.344045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8fc37-f9a2-4f1d-bbb4-8f2d3111e043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:34.344338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8fc37-f9a2-4f1d-bbb4-8f2d3111e043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:34.353736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8fc37-f9a2-4f1d-bbb4-8f2d3111e043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:34.355198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8fc37-f9a2-4f1d-bbb4-8f2d3111e043 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:34.364569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1303/2604] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:34.913020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:34.915045 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:34.926930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88e3fa-8828-4a5d-84a9-b06e7e964c3e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:34.932498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:34.935646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:34.935931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:34.936228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] PATH_INFO: `/v3/projects/9193c2d554e649d280251ad312e94ead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:34.937518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9193c2d554e649d280251ad312e94ead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:34.939522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.066208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9193c2d554e649d280251ad312e94ead', 'request_id': 'req-a2350b1e-6503-434a-a2d9-99ad0e230afc'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:35.133883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9193c2d554e649d280251ad312e94ead', 'request_id': 'req-a2350b1e-6503-434a-a2d9-99ad0e230afc'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:35.135208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:35.140233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9193c2d554e649d280251ad312e94ead', 'request_id': 'req-a2350b1e-6503-434a-a2d9-99ad0e230afc'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:35.151296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] The token cache is being invalidate because project 9193c2d554e649d280251ad312e94ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:35.151583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:35.152247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a2350b1e-6503-434a-a2d9-99ad0e230afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:35.153743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1302/2605] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:34 2022] DELETE /identity/v3/projects/9193c2d554e649d280251ad312e94ead => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:35.158656 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.159114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.159251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.179653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:35.180916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:35.239299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1304/2606] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:35.248097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.248586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.248723 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.267904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:35.269146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:35.277340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1303/2607] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:35.281520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.291937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696753cf-7961-4968-9e7e-f3f18e3f5ebb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:35.296660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:35.306949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.306949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.306949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.306949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:35.306949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:35.306949 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.312922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1305/2608] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:35.317145 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.326284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84567792-ae40-4e91-a5af-82fd76375ff7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:35.330905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:35.331553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.331830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.332101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.332774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:35.334658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.411863 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1304/2609] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:35 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:35.416642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.427004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93885442-cc17-4c1f-809a-0970a3bc7d12 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:35.484375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:35.484992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.485272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.485545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.486186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:35.488205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.581622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1306/2610] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:35 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:35.586215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.597013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1712b-d999-483f-b6c3-a7c036bf7049 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:35.602223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:35.602825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.603104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.603433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.604072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:35.605683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.615977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1305/2611] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:35.620788 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.630893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a61f-e119-45d1-a92c-481406e34347 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:35.636538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ced73-35ef-4c88-a10c-6f87310a0ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:35.637345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ced73-35ef-4c88-a10c-6f87310a0ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.637707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ced73-35ef-4c88-a10c-6f87310a0ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.637995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ced73-35ef-4c88-a10c-6f87310a0ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.638670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ced73-35ef-4c88-a10c-6f87310a0ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:35.640305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ced73-35ef-4c88-a10c-6f87310a0ab3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.654811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1307/2612] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:35.656090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.666329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa7411-51d3-41e6-96a3-9c73c0976cc5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:35.670967 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:35.671626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.671900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.672186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] PATH_INFO: `/v3/projects/ef1179ff1a1a4b73bc5af5b756aa8777/users/bf248baa83064fe5979a372ff726b8b3/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.687697 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1179ff1a1a4b73bc5af5b756aa8777, user_id=bf248baa83064fe5979a372ff726b8b3, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:35.690547 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:35.791777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1306/2613] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:35 2022] PUT /identity/v3/projects/ef1179ff1a1a4b73bc5af5b756aa8777/users/bf248baa83064fe5979a372ff726b8b3/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:35.801864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:35.802145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:35.802415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:35.820443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:35.839922 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] MFA Rules not processed for user `bf248baa83064fe5979a372ff726b8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:35.842256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:35.923392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1308/2614] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:35.934440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:35.936101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:36.074368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:36.077102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb3dc-a999-4c28-abae-95488c1ca115 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:36.082564 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8460c7-8ad4-4279-83eb-018399a98908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:36.083161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8460c7-8ad4-4279-83eb-018399a98908 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:36.083459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8460c7-8ad4-4279-83eb-018399a98908 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:36.083729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8460c7-8ad4-4279-83eb-018399a98908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:36.091809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8460c7-8ad4-4279-83eb-018399a98908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:36.093394 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8460c7-8ad4-4279-83eb-018399a98908 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:36.101874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1307/2615] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:40.314980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:40.394695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1423ce-3d58-45a1-a983-280c69806f6c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:40.399594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:40.400251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:40.400528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:40.401230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] PATH_INFO: `/v3/users/bf248baa83064fe5979a372ff726b8b3` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:40.401940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf248baa83064fe5979a372ff726b8b3)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:40.403925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:40.596598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf248baa83064fe5979a372ff726b8b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:40.685022 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf248baa83064fe5979a372ff726b8b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:40.690983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:40.693104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf248baa83064fe5979a372ff726b8b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:40.693741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf248baa83064fe5979a372ff726b8b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:40.694339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf248baa83064fe5979a372ff726b8b3'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:40.711950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1309/2616] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:40 2022] DELETE /identity/v3/users/bf248baa83064fe5979a372ff726b8b3 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:40.716522 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:40.716984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:40.717079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:40.745696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:40.746950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:40.804837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1308/2617] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:40.814384 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:40.815852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:40.880355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1ca08-30c4-4a02-9386-11abe4c78a01 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:40.884645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644f60c-b1ea-421d-bada-270645aca2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:40.885239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644f60c-b1ea-421d-bada-270645aca2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:40.885512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644f60c-b1ea-421d-bada-270645aca2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:40.885778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644f60c-b1ea-421d-bada-270645aca2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:40.894795 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644f60c-b1ea-421d-bada-270645aca2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:40.896254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644f60c-b1ea-421d-bada-270645aca2d6 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:40.905700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1310/2618] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:41.292377 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:41.293886 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:41.306376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40b615-000c-4294-91a3-ee733306f818 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:41.311238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:41.311919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:41.312249 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:41.312521 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] PATH_INFO: `/v3/projects/ef1179ff1a1a4b73bc5af5b756aa8777` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:41.314153 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1179ff1a1a4b73bc5af5b756aa8777)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:41.316346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:41.428037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1179ff1a1a4b73bc5af5b756aa8777', 'request_id': 'req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:41.484715 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1179ff1a1a4b73bc5af5b756aa8777', 'request_id': 'req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:41.490672 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1179ff1a1a4b73bc5af5b756aa8777', 'request_id': 'req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:41.501875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] The token cache is being invalidate because project ef1179ff1a1a4b73bc5af5b756aa8777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:41.502166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:41.502819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fef27e0c-5c28-42d6-9e26-fe04beeb72f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:41.504313 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1309/2619] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:41 2022] DELETE /identity/v3/projects/ef1179ff1a1a4b73bc5af5b756aa8777 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:41.509455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:41.510040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:41.510040 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:41.560443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:41.561885 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:41.628959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1311/2620] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:41.638038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:41.638326 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:41.638600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:41.659148 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:41.660369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:41.668707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1310/2621] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:41.672863 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:41.682431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c945b-5aa3-4c28-bc9f-29945039ecf2 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:41.686977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:41.687992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:41.687992 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:41.688186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:41.688829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:41.689130 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:41.690718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:41.702119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1312/2622] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:41.706414 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:41.716442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf466e9-4624-48be-a36e-fc6a1a4cd6bc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:41.721071 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:41.721664 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:41.721927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:41.722193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:41.722815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:41.724744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:41.869726 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1311/2623] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:41 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:41.873939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:41.884449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535ddab-9612-4ed3-8420-2d9469152104 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:41.926262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:41.941010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:41.941700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:41.941987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:41.942263 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:41.942900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:41.945455 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:42.022667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1313/2624] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:41 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:42.027416 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:42.038357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c47a9-3667-4c11-bedf-879938b5ca42 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:42.043702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:42.043702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:42.043938 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:42.044166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:42.044818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:42.046413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:42.056460 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1312/2625] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:42.060368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:42.070644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c11bf9-a70b-4bbb-9e14-5b676b5b5841 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:42.075277 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e5e3b-3142-4cf7-a122-bc9bf0f35e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:42.075862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5e3b-3142-4cf7-a122-bc9bf0f35e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:42.076162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5e3b-3142-4cf7-a122-bc9bf0f35e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:42.076448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5e3b-3142-4cf7-a122-bc9bf0f35e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:42.077084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e5e3b-3142-4cf7-a122-bc9bf0f35e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:42.078684 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e5e3b-3142-4cf7-a122-bc9bf0f35e36 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:42.089760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1314/2626] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:42.093627 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:42.103558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164f6ff-f8f1-494d-ad94-cebe6d0e60f0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:42.108097 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:42.108777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:42.109058 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:42.109332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] PATH_INFO: `/v3/projects/8d4bd5d1e14c4f9fab898d5a16fbc962/users/6899a6ebd0a349c7a20fcf2170f6c24a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:42.123736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4bd5d1e14c4f9fab898d5a16fbc962, user_id=6899a6ebd0a349c7a20fcf2170f6c24a, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:42.126355 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:42.190777 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1313/2627] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:42 2022] PUT /identity/v3/projects/8d4bd5d1e14c4f9fab898d5a16fbc962/users/6899a6ebd0a349c7a20fcf2170f6c24a/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:42.202013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:42.202300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:42.202575 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:42.229606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] MFA Rules not processed for user `6899a6ebd0a349c7a20fcf2170f6c24a`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:42.230812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:42.292953 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1315/2628] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:42.304286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:42.306490 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:42.311708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:42.378035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d10cf8-e1a4-4751-a7ad-a0dde22664ad admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:42.384660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076b524-781e-42cc-8949-1c8aa211d9b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:42.385258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076b524-781e-42cc-8949-1c8aa211d9b3 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:42.385536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076b524-781e-42cc-8949-1c8aa211d9b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:42.385806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076b524-781e-42cc-8949-1c8aa211d9b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:42.393310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076b524-781e-42cc-8949-1c8aa211d9b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:42.394872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076b524-781e-42cc-8949-1c8aa211d9b3 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:42.403693 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1314/2629] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:42.614147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:42.677610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45563c9-2958-4f72-a6be-6226571a41df None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:42.683716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:42.684512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:42.684793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:42.685067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] PATH_INFO: `/v3/users/6899a6ebd0a349c7a20fcf2170f6c24a` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:42.686234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6899a6ebd0a349c7a20fcf2170f6c24a)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:42.688218 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:42.990916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6899a6ebd0a349c7a20fcf2170f6c24a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.061957 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6899a6ebd0a349c7a20fcf2170f6c24a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.063203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:43.069507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6899a6ebd0a349c7a20fcf2170f6c24a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.070153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6899a6ebd0a349c7a20fcf2170f6c24a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.074621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6899a6ebd0a349c7a20fcf2170f6c24a'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.088282 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1316/2630] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:42 2022] DELETE /identity/v3/users/6899a6ebd0a349c7a20fcf2170f6c24a => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:43.092937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:43.093568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:43.094311 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:43.114740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:43.115962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:43.174169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1315/2631] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:43.183829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:43.185275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:43.242121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c719-fc47-4d81-9648-850d3a19d6f0 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:43.245640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0588-67d4-4555-955a-2f1f7f93be17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:43.246240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0588-67d4-4555-955a-2f1f7f93be17 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:43.246519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0588-67d4-4555-955a-2f1f7f93be17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:43.246796 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0588-67d4-4555-955a-2f1f7f93be17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:43.255640 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0588-67d4-4555-955a-2f1f7f93be17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:43.257109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0588-67d4-4555-955a-2f1f7f93be17 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:43.266440 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1317/2632] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:43.658038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:43.661749 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:43.672287 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66a37-31ce-4a6b-b6ac-291c11e5efdb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:43.677084 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:43.677868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:43.678215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:43.678416 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] PATH_INFO: `/v3/projects/8d4bd5d1e14c4f9fab898d5a16fbc962` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:43.680104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4bd5d1e14c4f9fab898d5a16fbc962)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:43.682193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:43.793286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4bd5d1e14c4f9fab898d5a16fbc962', 'request_id': 'req-ec9885af-1923-47d1-b876-24121529c43e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.868447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4bd5d1e14c4f9fab898d5a16fbc962', 'request_id': 'req-ec9885af-1923-47d1-b876-24121529c43e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.874604 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4bd5d1e14c4f9fab898d5a16fbc962', 'request_id': 'req-ec9885af-1923-47d1-b876-24121529c43e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.886211 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] The token cache is being invalidate because project 8d4bd5d1e14c4f9fab898d5a16fbc962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:43.886534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.887168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9885af-1923-47d1-b876-24121529c43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:43.888674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1316/2633] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:43 2022] DELETE /identity/v3/projects/8d4bd5d1e14c4f9fab898d5a16fbc962 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:43.893746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:43.894271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:43.894271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:43.914461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:43.915691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:43.982153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1318/2634] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:43.990895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:43.991246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:43.991653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.022013 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:44.024306 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:44.042867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1317/2635] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:44.047407 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.057835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e603f9-37e0-4a0c-99e4-a1dc0384a52d None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.062509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.063106 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.063388 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.063662 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.064339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:44.064642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.066244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.078453 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1319/2636] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:44.082801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.088447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:44.092494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307462-ba68-472c-bafe-848fcf105af0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.097472 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.098968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.098968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.098968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.099367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.101907 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.198511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1318/2637] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:44 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:44.202526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.212557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2f0d-e2ce-427c-b99c-d69fadbff00e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.271223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.272084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.272395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.272667 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.273349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.275275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.284267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:44.420955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1320/2638] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:44 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:53:44.425412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.436403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013512d-6c76-47fe-bbb1-4e43ee791ca4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.440975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.441580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.446874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.446874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.446874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.446874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.454266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1319/2639] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:44.458530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.469144 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b7944-7b19-400d-aad3-5bfd0246c486 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.473693 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50214c2d-a923-4ebf-b4c6-54c5a34059f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.474297 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50214c2d-a923-4ebf-b4c6-54c5a34059f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.474572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50214c2d-a923-4ebf-b4c6-54c5a34059f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.474839 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50214c2d-a923-4ebf-b4c6-54c5a34059f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.475466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50214c2d-a923-4ebf-b4c6-54c5a34059f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.477760 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50214c2d-a923-4ebf-b4c6-54c5a34059f7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.488095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1321/2640] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:44.492350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.502763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ba547-d8f1-49e5-8126-e682707ca5c1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.507477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.508223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.508503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.508778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] PATH_INFO: `/v3/projects/205dd498ed354ed8bce6f7f0e66c3947/users/de39947ed3aa4b5f8c9eb9582c5cc583/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.525221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205dd498ed354ed8bce6f7f0e66c3947, user_id=de39947ed3aa4b5f8c9eb9582c5cc583, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.527963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.645269 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1320/2641] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:44 2022] PUT /identity/v3/projects/205dd498ed354ed8bce6f7f0e66c3947/users/de39947ed3aa4b5f8c9eb9582c5cc583/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:44.657629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.657918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.658191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.687591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] MFA Rules not processed for user `de39947ed3aa4b5f8c9eb9582c5cc583`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:44.688773 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:44.753492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1322/2642] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:44.758165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.825010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00153e87-bba9-41bc-a957-bdc4911e5956 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:44.829694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:44.830298 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:44.830578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:44.830852 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:44.831484 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:44.833401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:44.979801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1321/2643] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:44 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:44.983902 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:44.994691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3f9e8-e8d3-4898-ac98-7e3d94331167 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:45.056309 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:45.056984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.057419 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.060286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.060286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:45.061705 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:45.171194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1323/2644] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:44 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:45.175266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:45.186384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144e9d7-4806-4e3f-b8d2-957d0ea5d1cd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:45.191253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:45.191714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.192169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.192537 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.193255 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:45.194923 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:45.208193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1322/2645] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:45.211190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:45.221413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4037e0-c195-4e46-baa1-5acee29cdb4c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:45.225988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:45.227256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.227533 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.227800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] PATH_INFO: `/v3/projects/55f0f5bcc3a24c7e8a7254e15bb6fce8/users/90b24328b25f4ccdb1713140b6af8990/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.242379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f0f5bcc3a24c7e8a7254e15bb6fce8, user_id=90b24328b25f4ccdb1713140b6af8990, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:45.245072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:45.340334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1324/2646] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:45 2022] PUT /identity/v3/projects/55f0f5bcc3a24c7e8a7254e15bb6fce8/users/90b24328b25f4ccdb1713140b6af8990/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:45.344404 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:45.355109 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e0f05-e1d2-45d4-b55e-e7ea00acdcf6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:45.416528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31665cf6-aa55-4dd9-9325-2bfbcb58328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:45.417280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31665cf6-aa55-4dd9-9325-2bfbcb58328c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.417560 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31665cf6-aa55-4dd9-9325-2bfbcb58328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.417829 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31665cf6-aa55-4dd9-9325-2bfbcb58328c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.418470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31665cf6-aa55-4dd9-9325-2bfbcb58328c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:45.420053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31665cf6-aa55-4dd9-9325-2bfbcb58328c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:45.430354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1323/2647] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:45.434711 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:45.445477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b2066-a608-4905-a41f-0716e5ee157c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:45.450264 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:45.450914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.451194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.451468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] PATH_INFO: `/v3/projects/55f0f5bcc3a24c7e8a7254e15bb6fce8/users/90b24328b25f4ccdb1713140b6af8990/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.453463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f0f5bcc3a24c7e8a7254e15bb6fce8, user_id=90b24328b25f4ccdb1713140b6af8990, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:45.456383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:45.467006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1325/2648] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:45 2022] PUT /identity/v3/projects/55f0f5bcc3a24c7e8a7254e15bb6fce8/users/90b24328b25f4ccdb1713140b6af8990/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:45.477696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.477919 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.478189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.505790 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] MFA Rules not processed for user `90b24328b25f4ccdb1713140b6af8990`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:45.506993 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:45.511909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:45.573074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1324/2649] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:45.583868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:45.586310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:45.611967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:45.642736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6fcc4-b856-403d-b6b1-ab81e198dd01 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:45.647146 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99823645-9baf-4c92-8f66-3c2565dc2524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:45.647735 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99823645-9baf-4c92-8f66-3c2565dc2524 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:45.648654 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99823645-9baf-4c92-8f66-3c2565dc2524 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:45.648941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99823645-9baf-4c92-8f66-3c2565dc2524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:45.657704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99823645-9baf-4c92-8f66-3c2565dc2524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:45.659166 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99823645-9baf-4c92-8f66-3c2565dc2524 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:45.668482 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1326/2650] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:46.781073 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:46.844020 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d53d7-8b5e-4744-bfc2-204821a384e9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:46.848553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:46.849271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:46.849554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:46.849825 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] PATH_INFO: `/v3/users/de39947ed3aa4b5f8c9eb9582c5cc583` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:46.850951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de39947ed3aa4b5f8c9eb9582c5cc583)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:46.852946 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:47.096065 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de39947ed3aa4b5f8c9eb9582c5cc583'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:47.096861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de39947ed3aa4b5f8c9eb9582c5cc583'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:47.097352 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de39947ed3aa4b5f8c9eb9582c5cc583'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:47.190619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de39947ed3aa4b5f8c9eb9582c5cc583'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:47.198056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de39947ed3aa4b5f8c9eb9582c5cc583'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:47.215446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1325/2651] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:46 2022] DELETE /identity/v3/users/de39947ed3aa4b5f8c9eb9582c5cc583 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:47.220129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:47.220634 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:47.220732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:47.240811 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:47.242833 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:47.301181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1327/2652] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:47.311373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:47.312795 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:47.339136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:47.370222 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b1cb-cbbd-4a69-b970-586120647b49 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:47.373887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:47.374584 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:47.374862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:47.375133 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:47.384546 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:47.386091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:47.395606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1326/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:47.879900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:47.881753 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:47.904824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba30782-9748-4530-a6a8-f98c010d17d4 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:47.912071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:47.912815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:47.915135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:47.918156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] PATH_INFO: `/v3/projects/205dd498ed354ed8bce6f7f0e66c3947` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:47.920755 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] RBAC: Authorizing `identity:delete_project(project_id=205dd498ed354ed8bce6f7f0e66c3947)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:47.923838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:48.070566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205dd498ed354ed8bce6f7f0e66c3947', 'request_id': 'req-80695beb-85a5-43e9-bdec-eadb3985dcff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.142505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205dd498ed354ed8bce6f7f0e66c3947', 'request_id': 'req-80695beb-85a5-43e9-bdec-eadb3985dcff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.152900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205dd498ed354ed8bce6f7f0e66c3947', 'request_id': 'req-80695beb-85a5-43e9-bdec-eadb3985dcff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.161581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] The token cache is being invalidate because project 205dd498ed354ed8bce6f7f0e66c3947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:48.161895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.162553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.164357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1328/2654] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:47 2022] DELETE /identity/v3/projects/205dd498ed354ed8bce6f7f0e66c3947 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:48.168101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:48.177714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeb2ff-0896-4c9a-a187-3edaa2eada5d service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:48.235462 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:48.236091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:48.236402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:48.236678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] PATH_INFO: `/v3/users/90b24328b25f4ccdb1713140b6af8990` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:48.237797 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b24328b25f4ccdb1713140b6af8990)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:48.239705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:48.447026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b24328b25f4ccdb1713140b6af8990'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.451197 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b24328b25f4ccdb1713140b6af8990'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.451832 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b24328b25f4ccdb1713140b6af8990'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.519362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b24328b25f4ccdb1713140b6af8990'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.527001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0e49aa4a-8436-4877-ab57-5308b994ec73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b24328b25f4ccdb1713140b6af8990'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:48.544026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1327/2655] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:48 2022] DELETE /identity/v3/users/90b24328b25f4ccdb1713140b6af8990 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:49.212823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:49.214487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:49.228627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80695beb-85a5-43e9-bdec-eadb3985dcff admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:49.274951 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:49.286258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:49.286880 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.287156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.287424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] PATH_INFO: `/v3/projects/55f0f5bcc3a24c7e8a7254e15bb6fce8` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.289217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f0f5bcc3a24c7e8a7254e15bb6fce8)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:49.294681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:49.422852 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f0f5bcc3a24c7e8a7254e15bb6fce8', 'request_id': 'req-ff091104-03c8-4940-8dad-4f7656dc83b8'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:49.487935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f0f5bcc3a24c7e8a7254e15bb6fce8', 'request_id': 'req-ff091104-03c8-4940-8dad-4f7656dc83b8'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:49.493758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f0f5bcc3a24c7e8a7254e15bb6fce8', 'request_id': 'req-ff091104-03c8-4940-8dad-4f7656dc83b8'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:49.505067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] The token cache is being invalidate because project 55f0f5bcc3a24c7e8a7254e15bb6fce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:49.505352 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:49.505982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ff091104-03c8-4940-8dad-4f7656dc83b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:49.507980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1329/2656] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:49 2022] DELETE /identity/v3/projects/55f0f5bcc3a24c7e8a7254e15bb6fce8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:49.513609 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.514101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.514202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.534284 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:49.535482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:49.575807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:49.594516 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1328/2657] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:49.602470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.602785 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.603087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.622707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:49.623899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:49.631802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1330/2658] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:49.636758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:49.646893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e35f58-96ec-4ff6-8f4f-ef5d4eed581f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:49.652039 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:49.652665 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.652939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.653264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.653901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:49.654199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:49.655778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:49.666966 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1329/2659] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 14:53:49.671285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:49.681094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322572c-4eef-4d28-b5d7-10743aab543f None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:49.685777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:49.686357 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.686631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.686898 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.687544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:49.689574 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:49.804690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1331/2660] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:49 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:49.810421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:49.822621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e650e-ff87-4282-b3d2-e2243717a407 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:49.879437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:49.880001 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.880309 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.880586 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.881257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:49.883736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:49.958199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1330/2661] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:49 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:53:49.964189 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:49.975861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2494b-a436-40bd-99a4-8cf241e2710f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:49.980692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:49.984235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:49.984235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:49.984235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:49.984235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:49.984235 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:49.995167 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1332/2662] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:49.999491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:50.010283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7324bd-e0f0-4e20-906b-9db2e5451844 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:50.015163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317a7b3-e2cd-485f-9e55-448a2158feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:50.015754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317a7b3-e2cd-485f-9e55-448a2158feb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.016029 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317a7b3-e2cd-485f-9e55-448a2158feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.016338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317a7b3-e2cd-485f-9e55-448a2158feb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.016976 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317a7b3-e2cd-485f-9e55-448a2158feb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:50.018594 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317a7b3-e2cd-485f-9e55-448a2158feb0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:50.029559 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1331/2663] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:50.033551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:50.043432 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d2061-13a0-4952-9f6f-bf4dcad95055 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:50.048057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:50.048730 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.049006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.049274 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] PATH_INFO: `/v3/projects/8f2beb5725444682b2ac17dd3a54c858/users/ac2a298ce0fb4ccfa14c3c656eb5f9ec/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.064225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2beb5725444682b2ac17dd3a54c858, user_id=ac2a298ce0fb4ccfa14c3c656eb5f9ec, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:50.066914 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:50.136198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1333/2664] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:50 2022] PUT /identity/v3/projects/8f2beb5725444682b2ac17dd3a54c858/users/ac2a298ce0fb4ccfa14c3c656eb5f9ec/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:50.146146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.146429 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.146951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.174493 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] MFA Rules not processed for user `ac2a298ce0fb4ccfa14c3c656eb5f9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:50.175696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:50.237207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1332/2665] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:50.248954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:50.251208 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:50.306935 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bec896-9072-4631-9358-40137986a13d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:50.310605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed285b-b4cc-4c97-bd24-db2dcee97bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:50.311200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed285b-b4cc-4c97-bd24-db2dcee97bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.311479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed285b-b4cc-4c97-bd24-db2dcee97bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.311750 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed285b-b4cc-4c97-bd24-db2dcee97bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.319929 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed285b-b4cc-4c97-bd24-db2dcee97bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:50.321410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed285b-b4cc-4c97-bd24-db2dcee97bb0 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:50.330035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1334/2666] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:50.353182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:50.395230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:50.415221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132bb46-e3de-41a9-9326-cc6f74cbcadd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:50.420062 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:50.420718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.420994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.421261 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] PATH_INFO: `/v3/users/ac2a298ce0fb4ccfa14c3c656eb5f9ec` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.422402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2a298ce0fb4ccfa14c3c656eb5f9ec)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:50.424421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:50.617562 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2a298ce0fb4ccfa14c3c656eb5f9ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:50.618256 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2a298ce0fb4ccfa14c3c656eb5f9ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:50.618871 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2a298ce0fb4ccfa14c3c656eb5f9ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:50.696477 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2a298ce0fb4ccfa14c3c656eb5f9ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:50.703844 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2a298ce0fb4ccfa14c3c656eb5f9ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:50.721356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1333/2667] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:50 2022] DELETE /identity/v3/users/ac2a298ce0fb4ccfa14c3c656eb5f9ec => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:50.733304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.733658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.733921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.752948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:50.754114 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:50.765886 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:50.813629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1335/2668] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:50.823895 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:50.825425 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:50.885611 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae907b-69fd-4b3b-98b7-06aa582ac9ff admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:50.889299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d7c6f-f50d-42d2-9b46-a44b74c7646d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:50.889902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d7c6f-f50d-42d2-9b46-a44b74c7646d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:50.890180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d7c6f-f50d-42d2-9b46-a44b74c7646d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:50.890449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d7c6f-f50d-42d2-9b46-a44b74c7646d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:50.900207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d7c6f-f50d-42d2-9b46-a44b74c7646d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:50.901638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d7c6f-f50d-42d2-9b46-a44b74c7646d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:50.910944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1334/2669] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:51.332019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:51.336013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:51.346177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5d931-cb4a-4bd5-8528-e42ec78bd957 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:51.350988 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:51.352318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:51.352600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:51.352975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] PATH_INFO: `/v3/projects/8f2beb5725444682b2ac17dd3a54c858` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:51.354111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2beb5725444682b2ac17dd3a54c858)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:51.356033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:51.492116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2beb5725444682b2ac17dd3a54c858', 'request_id': 'req-5f72b44c-5e1f-433e-adc6-617b2bb5f848'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:51.557002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2beb5725444682b2ac17dd3a54c858', 'request_id': 'req-5f72b44c-5e1f-433e-adc6-617b2bb5f848'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:51.563134 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2beb5725444682b2ac17dd3a54c858', 'request_id': 'req-5f72b44c-5e1f-433e-adc6-617b2bb5f848'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:51.574968 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] The token cache is being invalidate because project 8f2beb5725444682b2ac17dd3a54c858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:51.575258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:51.575909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5f72b44c-5e1f-433e-adc6-617b2bb5f848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:51.577979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1336/2670] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:51 2022] DELETE /identity/v3/projects/8f2beb5725444682b2ac17dd3a54c858 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:51.583550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:51.583833 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:51.584101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:51.604279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:51.605491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:51.690237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1335/2671] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:51.697998 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:51.698305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:51.698551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:51.720866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:51.728289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:51.730774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1337/2672] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:51.735232 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:51.740113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:51.745890 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9650f-e8a9-4d21-bde3-bea9d727a2a2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:51.750528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:51.751127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:51.751402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:51.751688 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:51.752368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:51.752668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:51.754271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:51.769862 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1336/2673] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:51.771224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:51.782006 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb1aee-4b99-4f1d-9478-624e6206c31e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:51.787016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:51.787812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:51.788109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:51.788418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:51.789116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:51.791333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:51.926509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1338/2674] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:51 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:51.927621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:51.938997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5062664-43fc-4018-9cd9-c9ad0aa48f48 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.012557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.013166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.013445 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.013722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.014361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.016345 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.131209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1337/2675] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:51 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:52.135944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:52.146505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00893d1-e7cc-44a9-a529-38d12fe37ece admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.151721 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.152458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.152803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.153177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.153882 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.155624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.166017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1339/2676] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:52.169990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:52.180815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e8437-9e14-439f-9c62-e2b91920f888 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.187579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.188295 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.188502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.188769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.189421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.191184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.208246 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1338/2677] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:52.209109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:52.219630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb7d40-3ba9-4e00-b06f-700de449178e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.225507 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec1ca5-b8fe-41a9-b621-bd6df365de00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.226240 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec1ca5-b8fe-41a9-b621-bd6df365de00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.226522 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec1ca5-b8fe-41a9-b621-bd6df365de00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.226800 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec1ca5-b8fe-41a9-b621-bd6df365de00 admin admin] PATH_INFO: `/v3/projects/2710cdaf256342faae76fd5451ed432b/users/c18dbeea047a4505b2d5417dc54e8cb4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.242732 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec1ca5-b8fe-41a9-b621-bd6df365de00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2710cdaf256342faae76fd5451ed432b, user_id=c18dbeea047a4505b2d5417dc54e8cb4, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.245442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec1ca5-b8fe-41a9-b621-bd6df365de00 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.312861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:52.314963 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:52.378584 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1340/2678] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:52 2022] PUT /identity/v3/projects/2710cdaf256342faae76fd5451ed432b/users/c18dbeea047a4505b2d5417dc54e8cb4/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:52.386084 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.386380 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.386664 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.406223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:52.408231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:52.413342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2a898-f45f-4574-b774-06318c3b2ac8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.444449 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:52.465602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:52.474620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1341/2679] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:52.477985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:52.477985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:52.484810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.491444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.491444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.491444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.491444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:52.491983 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d4f9a-b697-4f38-95fa-a7c4e74e3e10 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.497057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.497652 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.497927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.498194 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.500290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.501902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.506691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.508803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.513292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1339/2680] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:53:52.514176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:52.516107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:52.520002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1342/2681] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:52.573018 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206159b-2e99-4ed2-9c94-8671b98ae360 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:52.576671 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:52.577276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:52.577550 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:52.577818 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:52.587200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:52.588800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:52.598314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1340/2682] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:54.192677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:54.205009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa3bc6-5b0a-4285-b256-24237fbfe9c3 service placement] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:54.209817 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:54.211302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:54.211799 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:54.212349 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:54.214603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:54.228557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:54.231224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:54.253464 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1343/2683] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:54.336215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:54.345677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26562-c317-4144-8976-bbf864af504d service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:54.349371 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:54.349984 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:54.350263 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:54.350534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:54.373117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:54.464362 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:54.465758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519fa97-294e-435c-b6f8-6c75a85d4339 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:54.477932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1341/2684] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:56.324829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:56.326904 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:56.343994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13328b-4b3f-4634-8556-1385b9af9f2d service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:56.350864 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:56.351931 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:56.352458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:56.352913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] PATH_INFO: `/v3/users/ff43bfa4484e4c1889d66eb9aab61374` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:56.354342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff43bfa4484e4c1889d66eb9aab61374)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:56.356598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:56.547019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff43bfa4484e4c1889d66eb9aab61374'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:56.622776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff43bfa4484e4c1889d66eb9aab61374'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:56.638394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff43bfa4484e4c1889d66eb9aab61374'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:56.640598 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff43bfa4484e4c1889d66eb9aab61374'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:56.641483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff43bfa4484e4c1889d66eb9aab61374'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:56.665474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1344/2685] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:53:56 2022] DELETE /identity/v3/users/ff43bfa4484e4c1889d66eb9aab61374 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:56.671601 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:56.672733 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:56.673187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:56.696578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:56.698329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:56.819243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1342/2686] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:56.834198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:56.836523 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:56.944213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4b79c-f10a-4338-80f2-5319a4538801 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:56.949019 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ec9de-2166-4a44-a90c-098fd1ac27fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:56.949725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ec9de-2166-4a44-a90c-098fd1ac27fe service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:56.950132 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ec9de-2166-4a44-a90c-098fd1ac27fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:56.951222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ec9de-2166-4a44-a90c-098fd1ac27fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:56.969137 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ec9de-2166-4a44-a90c-098fd1ac27fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:56.970672 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ec9de-2166-4a44-a90c-098fd1ac27fe service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:56.983687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1345/2687] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:53:57.731113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:57.732616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:57.758849 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4186a-0f0e-4491-8ec9-8b9997773ffd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:57.766038 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:57.766780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:57.766950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:57.767226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] PATH_INFO: `/v3/projects/7128afc4275e47dfbc5d8fb5d49ea58a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:57.770994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7128afc4275e47dfbc5d8fb5d49ea58a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:57.776202 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:57.780190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:57.911195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7128afc4275e47dfbc5d8fb5d49ea58a', 'request_id': 'req-b591b275-8ba4-4078-9bed-327fbba05ba1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:57.999647 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7128afc4275e47dfbc5d8fb5d49ea58a', 'request_id': 'req-b591b275-8ba4-4078-9bed-327fbba05ba1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:58.008653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7128afc4275e47dfbc5d8fb5d49ea58a', 'request_id': 'req-b591b275-8ba4-4078-9bed-327fbba05ba1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:58.021526 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] The token cache is being invalidate because project 7128afc4275e47dfbc5d8fb5d49ea58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:53:58.022401 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:58.023935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b591b275-8ba4-4078-9bed-327fbba05ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:53:58.030530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1343/2688] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:53:57 2022] DELETE /identity/v3/projects/7128afc4275e47dfbc5d8fb5d49ea58a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:58.032534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.032534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.032534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.048504 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:58.055747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:58.057258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:58.163481 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1346/2689] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:58.171147 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.171471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.171800 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.192893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:58.194230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:58.206031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1344/2690] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:58.210099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.225121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b8d1-bbe8-4ca2-b29f-9846094aebe5 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:58.231212 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:58.231847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.232109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.232457 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.233120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:53:58.233568 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:58.235682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:58.251131 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1347/2691] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:53:58.255400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.270403 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd62bd3-7029-4ea6-bcfa-82673913513f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:58.277700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:58.277700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.277700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.278108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.278428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:58.281143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:58.387037 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1345/2692] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:58.387479 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.401624 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ac385-872d-45b3-86ec-3c8db83aac60 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:58.481450 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:58.482231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.482527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.482807 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.483520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:58.487979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:58.618228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1348/2693] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:58.623680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.636822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4295e-c982-4130-95ae-ad8336df3512 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:58.642679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce34c09-75c4-485c-a140-a8d2ce39a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:58.643365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce34c09-75c4-485c-a140-a8d2ce39a187 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.643690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce34c09-75c4-485c-a140-a8d2ce39a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.644229 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce34c09-75c4-485c-a140-a8d2ce39a187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.645645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce34c09-75c4-485c-a140-a8d2ce39a187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:58.648956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce34c09-75c4-485c-a140-a8d2ce39a187 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:58.661207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1346/2694] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:58.666221 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.677323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a0a9-2fd7-41f9-843b-7d07ab886e07 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:58.685987 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:58.687292 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.687636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.687911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.688604 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:58.690551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:58.711401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1349/2695] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:58.716486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e43f9-7175-4f8a-9a74-c3f0bab61ccc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.716768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e43f9-7175-4f8a-9a74-c3f0bab61ccc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.717093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e43f9-7175-4f8a-9a74-c3f0bab61ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.717805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.729223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e51229-5489-4f79-bc3e-c31b7ad3074c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:58.735401 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:58.736065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.738549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.738549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] PATH_INFO: `/v3/projects/adcebece6a3347908f81eac933c82cfd/users/115648bd605144a6b896a6fb737c3949/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.755063 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adcebece6a3347908f81eac933c82cfd, user_id=115648bd605144a6b896a6fb737c3949, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:58.758149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:58.777927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-435e43f9-7175-4f8a-9a74-c3f0bab61ccc None None] MFA Rules not processed for user `c18dbeea047a4505b2d5417dc54e8cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:58.779170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-435e43f9-7175-4f8a-9a74-c3f0bab61ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:58.850687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1350/2696] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:58 2022] PUT /identity/v3/projects/adcebece6a3347908f81eac933c82cfd/users/115648bd605144a6b896a6fb737c3949/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:58.857668 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1347/2697] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:58.862328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:58.864195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:58.926373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:58.926670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:58.926947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:58.958528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:58.981767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] MFA Rules not processed for user `115648bd605144a6b896a6fb737c3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:58.983091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:58.996791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ecd74-0ed9-4dfe-98a1-9fafad705529 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.002660 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.003287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.003636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.004286 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.004536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.006474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.024663 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:59.061816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1348/2698] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:53:59.069402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.080853 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfb432-d4a3-4a5c-8061-92e5734e95ca None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.085728 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.086346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.086621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.086893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.087529 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.089478 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.116394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1351/2699] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:58 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:59.116394 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.220846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1349/2700] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:59 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:59.221802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.231784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5b74c-a28d-43d8-8b01-881341d6dae9 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.358573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.359835 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.360322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.360605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.361300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.363237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460af1-ee7a-4296-9b59-123ffe13cbe5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.363426 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.369597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.370230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.370711 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.371047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.371737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.374138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.491709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1352/2701] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:59 2022] POST /identity/v3/users => generated 298 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:59.495778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.503335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1350/2702] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:59 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:53:59.506555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf384019-f2b5-4384-bc64-804b54e5b24b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.507142 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.512177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.512763 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.513045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.513316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.513974 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.515607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.518690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64298d95-8a5a-49eb-b88b-1260cb214168 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.531546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1353/2703] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:59.531781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.531781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.531781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.531781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.532199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.532687 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.533971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.547979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16b519-4440-4c21-9c8b-6b7fc4ef91cd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.552279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1351/2704] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:59.553438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.554303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.554759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.555087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] PATH_INFO: `/v3/projects/e572a18003fb4d80982b0349bf8490ae/users/3bdb3970954a4f979f0c470ac7857254/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.555977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.567203 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13393-7476-46e5-abc4-5d956c5f0a05 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.572662 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.576491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.576491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.576491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.576491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.576491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.583778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e572a18003fb4d80982b0349bf8490ae, user_id=3bdb3970954a4f979f0c470ac7857254, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.588056 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.591528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1352/2705] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:53:59.593128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.603997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cedd6b-7cdd-4a80-91e0-dc77ce02e9aa admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.608910 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26a3fe-5ab8-4c44-9073-a43e0cc20e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.609588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26a3fe-5ab8-4c44-9073-a43e0cc20e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.609918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26a3fe-5ab8-4c44-9073-a43e0cc20e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.610191 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26a3fe-5ab8-4c44-9073-a43e0cc20e42 admin admin] PATH_INFO: `/v3/projects/9975990ba4e74749bbba9536fab924bc/users/fa90bfefb57e43efaf3a8833bd27d049/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.626727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26a3fe-5ab8-4c44-9073-a43e0cc20e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9975990ba4e74749bbba9536fab924bc, user_id=fa90bfefb57e43efaf3a8833bd27d049, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.629433 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26a3fe-5ab8-4c44-9073-a43e0cc20e42 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.663165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1354/2706] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:59 2022] PUT /identity/v3/projects/e572a18003fb4d80982b0349bf8490ae/users/3bdb3970954a4f979f0c470ac7857254/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:53:59.667806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.697822 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1353/2707] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:59 2022] PUT /identity/v3/projects/9975990ba4e74749bbba9536fab924bc/users/fa90bfefb57e43efaf3a8833bd27d049/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:59.710920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.711206 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.711479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.719492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:59.744002 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] MFA Rules not processed for user `fa90bfefb57e43efaf3a8833bd27d049`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:53:59.745505 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:53:59.748404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e805a1-9d6f-4557-9666-c08ffe6987dd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.790236 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:53:59.807810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.808505 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.808901 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.809333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.809986 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.811956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.813808 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1354/2708] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:53:59.821718 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.825948 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1355/2709] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:53:59.830650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.835796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f7952-22c7-4143-adaa-e446e14b5a35 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.840567 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.845044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.845044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.845044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.845312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a50f3-4fb5-45f7-a0a9-25769348e627 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:53:59.845428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.845428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.847339 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:53:59.848116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:53:59.848452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:53:59.848734 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] PATH_INFO: `/v3/projects/e572a18003fb4d80982b0349bf8490ae/users/3bdb3970954a4f979f0c470ac7857254/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:53:59.850744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e572a18003fb4d80982b0349bf8490ae, user_id=3bdb3970954a4f979f0c470ac7857254, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:53:59.853696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:53:59.864171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1356/2710] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:53:59 2022] PUT /identity/v3/projects/e572a18003fb4d80982b0349bf8490ae/users/3bdb3970954a4f979f0c470ac7857254/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:53:59.918518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1355/2711] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:53:59 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:53:59.919767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:53:59.985526 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989e3a6-ae52-4caf-98d3-21ab2d20e5f1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:00.053775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:00.066149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.066149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.066149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.066149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:00.066149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:00.254362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1357/2712] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:53:59 2022] POST /identity/v3/users => generated 295 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:54:00.254874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:00.259279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdff003-02c1-42e8-82b5-22a279873e90 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:00.264017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:00.264669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.264969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.265331 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.266139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:00.267667 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:00.286213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1356/2713] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:54:00.289413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:00.301467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6352a-1f1f-4559-bee3-35dc00d052f2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:00.307467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:00.308503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.324812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.324812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] PATH_INFO: `/v3/projects/68b55cea87fb4108a0e265d4c20f743e/users/b3e9b737ccc742d39eac505a7e5a0a7b/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.337065 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b55cea87fb4108a0e265d4c20f743e, user_id=b3e9b737ccc742d39eac505a7e5a0a7b, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:00.340369 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:00.467493 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1358/2714] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:54:00 2022] PUT /identity/v3/projects/68b55cea87fb4108a0e265d4c20f743e/users/b3e9b737ccc742d39eac505a7e5a0a7b/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:00.472666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:00.490015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e5ef3-0e65-4a9a-b37a-124d73d44f84 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:00.553924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51623e73-6289-40d0-b0d8-30a34ab14416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:00.554645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623e73-6289-40d0-b0d8-30a34ab14416 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.554921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623e73-6289-40d0-b0d8-30a34ab14416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.555195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623e73-6289-40d0-b0d8-30a34ab14416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.555927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51623e73-6289-40d0-b0d8-30a34ab14416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:00.562113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51623e73-6289-40d0-b0d8-30a34ab14416 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:00.572928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1357/2715] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:54:00.574017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:00.589389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b582-fce5-4751-9188-25612e9c0224 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:00.608403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:00.609191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.609562 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.609846 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] PATH_INFO: `/v3/projects/68b55cea87fb4108a0e265d4c20f743e/users/b3e9b737ccc742d39eac505a7e5a0a7b/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.615467 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b55cea87fb4108a0e265d4c20f743e, user_id=b3e9b737ccc742d39eac505a7e5a0a7b, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:00.618410 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:00.644164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:00.664215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1359/2716] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:54:00 2022] PUT /identity/v3/projects/68b55cea87fb4108a0e265d4c20f743e/users/b3e9b737ccc742d39eac505a7e5a0a7b/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:54:00.666389 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.666855 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.667682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.699200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] MFA Rules not processed for user `b3e9b737ccc742d39eac505a7e5a0a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:00.700554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:00.797835 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1358/2717] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:00.810764 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:00.815088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:00.886916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9e389-75ef-4585-a873-98509f4b9e30 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:00.891252 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:00.892096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:00.892612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:00.892984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:00.963622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:00.965746 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:00.980345 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1360/2718] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:54:03.103753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:03.105195 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:03.152064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cb3d9-ec37-489d-bbee-cb554ab3313e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:03.160373 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c603-8beb-4cba-88f9-147bbc2a7021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:03.161503 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c603-8beb-4cba-88f9-147bbc2a7021 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:03.162012 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c603-8beb-4cba-88f9-147bbc2a7021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:03.162504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c603-8beb-4cba-88f9-147bbc2a7021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:03.177072 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:03.283177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20c603-8beb-4cba-88f9-147bbc2a7021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:03.284636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20c603-8beb-4cba-88f9-147bbc2a7021 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:03.296335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1359/2719] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:04.992225 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:05.011678 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fd016-ce3c-4027-b5da-fcbfe8926724 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:05.017236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:05.017236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:05.017236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:05.017236 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:05.027798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:05.029498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:05.040334 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1361/2720] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:06.422314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c70cb6-a6ad-4e7f-a779-b19917473a81 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:06.422702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c70cb6-a6ad-4e7f-a779-b19917473a81 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:06.422891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c70cb6-a6ad-4e7f-a779-b19917473a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:06.460325 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-17c70cb6-a6ad-4e7f-a779-b19917473a81 None None] MFA Rules not processed for user `3bdb3970954a4f979f0c470ac7857254`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:06.461653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-17c70cb6-a6ad-4e7f-a779-b19917473a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:06.542583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1360/2721] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:06.557328 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:06.558825 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:06.620191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd78bf9-8b89-4efc-9351-d6587923552f service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:06.624217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:06.625126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:06.626013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:06.626013 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:06.636806 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:06.698544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:06.700100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:06.709400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1362/2722] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:07.020540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:07.020540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:07.020540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:07.048358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:07.049880 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:07.114573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1361/2723] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:07.287545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:07.290616 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:07.377032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ae753-383d-4198-8a38-c20da35e74b0 service glance] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:07.387171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a0373-2053-4b07-b1d0-e9c7b9387717 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:07.387171 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a0373-2053-4b07-b1d0-e9c7b9387717 service swift] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:07.387759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a0373-2053-4b07-b1d0-e9c7b9387717 service swift] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:07.387759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a0373-2053-4b07-b1d0-e9c7b9387717 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:07.399636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a0373-2053-4b07-b1d0-e9c7b9387717 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:07.401315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a0373-2053-4b07-b1d0-e9c7b9387717 service swift] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:07.410149 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1363/2724] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:54:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:54:10.067107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:10.117369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:10.189837 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8d8a0-c842-4d34-bde8-b71defa09954 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:10.195482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:10.196450 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:10.196969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:10.197464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] PATH_INFO: `/v3/users/115648bd605144a6b896a6fb737c3949` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:10.200177 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=115648bd605144a6b896a6fb737c3949)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:10.202952 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:10.399995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115648bd605144a6b896a6fb737c3949'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:10.400714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115648bd605144a6b896a6fb737c3949'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:10.401327 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115648bd605144a6b896a6fb737c3949'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:10.482587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115648bd605144a6b896a6fb737c3949'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:10.491737 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115648bd605144a6b896a6fb737c3949'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:10.521467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1362/2725] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:54:10 2022] DELETE /identity/v3/users/115648bd605144a6b896a6fb737c3949 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:10.523878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:10.524399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:10.524668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:10.548991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:10.550514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:10.625812 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1364/2726] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:54:10.634571 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:10.636913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:10.715716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc876f-6d20-4437-a66b-df7e970ac540 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:10.722782 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:10.723453 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:10.723724 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:10.723998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:10.736209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:10.737896 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:10.771294 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1363/2727] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:11.592707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:11.592707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:11.606891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd715e7b-c5bc-4f30-b1fb-f601fa697bbb None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:11.611980 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:11.612703 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:11.613233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:11.613862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] PATH_INFO: `/v3/projects/adcebece6a3347908f81eac933c82cfd` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:11.615284 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adcebece6a3347908f81eac933c82cfd)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:11.617985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:11.757484 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:11.770837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adcebece6a3347908f81eac933c82cfd', 'request_id': 'req-3a385d86-529f-44ca-a8c5-23d16f61ce69'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:11.847971 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adcebece6a3347908f81eac933c82cfd', 'request_id': 'req-3a385d86-529f-44ca-a8c5-23d16f61ce69'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:11.856018 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adcebece6a3347908f81eac933c82cfd', 'request_id': 'req-3a385d86-529f-44ca-a8c5-23d16f61ce69'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:11.868186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] The token cache is being invalidate because project adcebece6a3347908f81eac933c82cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:54:11.868804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:11.869731 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:11.871842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1365/2728] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:54:11 2022] DELETE /identity/v3/projects/adcebece6a3347908f81eac933c82cfd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:11.876381 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:11.887882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ffdf-c9f7-47af-bd25-4bad23a4d09d service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:11.945921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:11.955792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:11.956466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:11.956745 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:11.957031 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] PATH_INFO: `/v3/users/fa90bfefb57e43efaf3a8833bd27d049` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:11.958178 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa90bfefb57e43efaf3a8833bd27d049)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:11.960893 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:12.139467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa90bfefb57e43efaf3a8833bd27d049'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:12.140171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa90bfefb57e43efaf3a8833bd27d049'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:12.140956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa90bfefb57e43efaf3a8833bd27d049'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:12.209332 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa90bfefb57e43efaf3a8833bd27d049'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:12.219867 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa90bfefb57e43efaf3a8833bd27d049'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:12.247944 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1364/2729] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:54:11 2022] DELETE /identity/v3/users/fa90bfefb57e43efaf3a8833bd27d049 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:12.762229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:12.764497 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:12.812436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a385d86-529f-44ca-a8c5-23d16f61ce69 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:12.895299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:12.896121 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:12.897094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:12.897635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] PATH_INFO: `/v3/projects/9975990ba4e74749bbba9536fab924bc` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:12.900089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9975990ba4e74749bbba9536fab924bc)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:12.903761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:13.049776 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9975990ba4e74749bbba9536fab924bc', 'request_id': 'req-b51e9e57-463f-4aa1-a19b-251dd17aea80'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.134379 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9975990ba4e74749bbba9536fab924bc', 'request_id': 'req-b51e9e57-463f-4aa1-a19b-251dd17aea80'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.140798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9975990ba4e74749bbba9536fab924bc', 'request_id': 'req-b51e9e57-463f-4aa1-a19b-251dd17aea80'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.155017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] The token cache is being invalidate because project 9975990ba4e74749bbba9536fab924bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:54:13.155525 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.156406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.158140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1366/2730] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:54:12 2022] DELETE /identity/v3/projects/9975990ba4e74749bbba9536fab924bc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:13.162497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:13.164250 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:13.179391 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d6f7b-15b6-4a2e-a227-ca88f14ee937 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:13.243155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:13.244551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:13.244990 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:13.245275 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] PATH_INFO: `/v3/users/b3e9b737ccc742d39eac505a7e5a0a7b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:13.246459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e9b737ccc742d39eac505a7e5a0a7b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:13.248444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:13.453348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e9b737ccc742d39eac505a7e5a0a7b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.454182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e9b737ccc742d39eac505a7e5a0a7b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.463474 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e9b737ccc742d39eac505a7e5a0a7b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.548638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e9b737ccc742d39eac505a7e5a0a7b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.565763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9133e44b-a238-4c68-8d02-870e443225f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e9b737ccc742d39eac505a7e5a0a7b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:13.577351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:13.599112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1365/2731] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:54:13 2022] DELETE /identity/v3/users/b3e9b737ccc742d39eac505a7e5a0a7b => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:14.144642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:14.147253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:14.182644 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9e57-463f-4aa1-a19b-251dd17aea80 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:14.257843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:14.268879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:14.269588 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:14.269869 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:14.270140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] PATH_INFO: `/v3/projects/68b55cea87fb4108a0e265d4c20f743e` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:14.271647 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b55cea87fb4108a0e265d4c20f743e)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:14.274351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:14.725300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b55cea87fb4108a0e265d4c20f743e', 'request_id': 'req-a4362341-bdd8-4098-b8b3-d73c394ab931'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:14.816476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b55cea87fb4108a0e265d4c20f743e', 'request_id': 'req-a4362341-bdd8-4098-b8b3-d73c394ab931'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:14.825163 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b55cea87fb4108a0e265d4c20f743e', 'request_id': 'req-a4362341-bdd8-4098-b8b3-d73c394ab931'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:14.840542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] The token cache is being invalidate because project 68b55cea87fb4108a0e265d4c20f743e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:54:14.841202 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:14.843183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-a4362341-bdd8-4098-b8b3-d73c394ab931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:14.845321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1367/2732] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:54:14 2022] DELETE /identity/v3/projects/68b55cea87fb4108a0e265d4c20f743e => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:14.850015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:14.850981 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:14.851069 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:14.877652 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:14.879000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:14.956317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1366/2733] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:14.962996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:14.963476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:14.963758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:14.993029 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:14.994200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:15.004782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1368/2734] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:54:15.009122 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.019143 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f2ad1-9f5e-4f37-8007-f12c8c351b1e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.024511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.025130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.025412 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.025683 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.026357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:54:15.026661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.028289 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.044208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1367/2735] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:54:15.048107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.061069 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4180bb7-5c1c-4b31-bbab-885aa6d33c4c None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.066233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.066856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.067135 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.067406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.068044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.070046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.173447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1369/2736] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:54:15 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:54:15.175359 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.188121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905e162-eeab-41b2-98f1-45ca5e160f9a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.263483 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.264485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.264971 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.265399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.266198 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.268329 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.358251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1368/2737] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:54:15 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:54:15.359502 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.374027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b370ccd-4d9f-47b8-9ae7-1b96dfc1d51f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.379312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.380517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.380967 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.381404 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.382438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.384181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.405601 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1370/2738] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:54:15.407114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.421996 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994c3a7-b9a2-4e35-99d5-12c1b46aa080 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.427193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ded47-a325-4ea3-bf2b-0fba925391b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.427878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ded47-a325-4ea3-bf2b-0fba925391b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.428168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ded47-a325-4ea3-bf2b-0fba925391b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.428443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ded47-a325-4ea3-bf2b-0fba925391b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.429128 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ded47-a325-4ea3-bf2b-0fba925391b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.431508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ded47-a325-4ea3-bf2b-0fba925391b5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.445758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1369/2739] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:54:15.450184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.463761 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f307f9-8022-42e1-a59f-a0b983ee0cb4 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.469307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.470759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.471140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.471535 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] PATH_INFO: `/v3/projects/d7dac3428efc4f3cbcae424f34977d8b/users/c1004bb147c1447ba5b9c85c527907fb/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.490591 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7dac3428efc4f3cbcae424f34977d8b, user_id=c1004bb147c1447ba5b9c85c527907fb, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.494048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.576211 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1371/2740] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:54:15 2022] PUT /identity/v3/projects/d7dac3428efc4f3cbcae424f34977d8b/users/c1004bb147c1447ba5b9c85c527907fb/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:54:15.586754 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137597-17d5-4ee0-a76a-8a550b86c17c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.588214 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137597-17d5-4ee0-a76a-8a550b86c17c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.588566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137597-17d5-4ee0-a76a-8a550b86c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.620975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-17137597-17d5-4ee0-a76a-8a550b86c17c None None] MFA Rules not processed for user `c1004bb147c1447ba5b9c85c527907fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:15.622310 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-17137597-17d5-4ee0-a76a-8a550b86c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:15.678292 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:15.699819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1370/2741] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:15.715939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:15.717713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:15.794304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcdc34-a1c9-4775-a0a3-a0fc14b30574 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:15.798682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037457c-3c1e-4e8a-8fdb-032796c82067 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '113ccc244a814d7580d1c9f6319edf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:15.799551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037457c-3c1e-4e8a-8fdb-032796c82067 service glance] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:15.799767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037457c-3c1e-4e8a-8fdb-032796c82067 service glance] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:15.800038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037457c-3c1e-4e8a-8fdb-032796c82067 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:15.810133 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037457c-3c1e-4e8a-8fdb-032796c82067 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:15.811590 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037457c-3c1e-4e8a-8fdb-032796c82067 service glance] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:15.823196 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1372/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:18.998536 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:18.999027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:18.999115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:19.021228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] MFA Rules not processed for user `c18dbeea047a4505b2d5417dc54e8cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:19.023383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:19.086016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1371/2743] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:22.859185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:22.859682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:22.859810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:22.881129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:22.882538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:22.904749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:22.941805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1373/2744] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Sep 7 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:54:22.954674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:22.956102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:23.033805 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7573c8-2f09-4077-b2a8-036f5145f699 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:23.039743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:23.040193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:23.040471 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:23.040742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:23.050094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:23.051666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:23.063608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1372/2745] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:54:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:54:33.056651 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=63 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:33.105126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=14 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:49.153156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:49.154572 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:49.211242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a956cec-38e7-4647-953d-cc03ca75c43a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:49.214748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:49.215354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:49.215622 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:49.215895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:49.217638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:49.279707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:49.281395 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fabab8-437a-49dd-929c-64fbeddb9292 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:49.291318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1374/2746] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:53.361613 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:53.432540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a6307-e8ef-4fb8-b73e-619bd33dd73d service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:53.437115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:53.437836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:53.438108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:53.438378 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] PATH_INFO: `/v3/users/c1004bb147c1447ba5b9c85c527907fb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:53.439494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1004bb147c1447ba5b9c85c527907fb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:53.441424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:53.664092 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1004bb147c1447ba5b9c85c527907fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:53.664731 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1004bb147c1447ba5b9c85c527907fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:53.665351 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1004bb147c1447ba5b9c85c527907fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:53.762156 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1004bb147c1447ba5b9c85c527907fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:53.770200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1004bb147c1447ba5b9c85c527907fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:53.789385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1373/2747] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:54:53 2022] DELETE /identity/v3/users/c1004bb147c1447ba5b9c85c527907fb => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:53.794665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:53.794665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:53.794665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:53.814462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:53.815670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:53.874355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1375/2748] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:53.883980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:53.885446 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:53.945830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a2102-8197-4ea3-a02e-fc1d5b509348 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:53.949482 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fe06d-637b-4e3f-b422-38c5c4285ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:53.950088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fe06d-637b-4e3f-b422-38c5c4285ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:53.950360 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fe06d-637b-4e3f-b422-38c5c4285ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:53.950628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fe06d-637b-4e3f-b422-38c5c4285ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:53.959768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fe06d-637b-4e3f-b422-38c5c4285ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:53.961209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fe06d-637b-4e3f-b422-38c5c4285ba7 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:53.974085 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1374/2749] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:54:54.371710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:54.373154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:54.385346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c0b7d-4eb8-47e9-bc71-88b205631649 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:54.389995 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:54.390628 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:54.390906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:54.391176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] PATH_INFO: `/v3/projects/d7dac3428efc4f3cbcae424f34977d8b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:54.402944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7dac3428efc4f3cbcae424f34977d8b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:54.402944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:54.533169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7dac3428efc4f3cbcae424f34977d8b', 'request_id': 'req-fb4615c4-3af7-4670-85a5-1a126ab41b76'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:54.535230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:54.622096 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7dac3428efc4f3cbcae424f34977d8b', 'request_id': 'req-fb4615c4-3af7-4670-85a5-1a126ab41b76'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:54.628471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7dac3428efc4f3cbcae424f34977d8b', 'request_id': 'req-fb4615c4-3af7-4670-85a5-1a126ab41b76'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:54.640630 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] The token cache is being invalidate because project d7dac3428efc4f3cbcae424f34977d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:54:54.640785 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:54.641465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-fb4615c4-3af7-4670-85a5-1a126ab41b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:54:54.642950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1376/2750] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:54:54 2022] DELETE /identity/v3/projects/d7dac3428efc4f3cbcae424f34977d8b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:54:54.648684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:54.649288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:54.649288 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:54.668783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:54.670053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:54.686713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:54.730374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1375/2751] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:54.738570 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:54.738855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:54.739126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:54.760700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:54.762002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:54.770087 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1377/2752] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:54:54.775226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:54.785452 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e6003-902c-4601-9d70-16649f6ddae6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:54.790372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:54.790975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:54.791257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:54.791532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:54.792201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:54:54.792508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:54.794131 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:54.805625 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1376/2753] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:54:54.809940 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:54.819872 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8813e-d9ef-4024-93b7-cdc0a549dc04 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:54.824666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:54.825256 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:54.825534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:54.825803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:54.826436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:54.828460 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:54.940917 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1378/2754] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:54:54 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:54:54.945646 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:54.955511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5c150-b1b1-4dac-ba58-da5f9b59e2c7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:55.013635 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:55.014237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:55.014514 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:55.014783 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:55.015398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:55.017349 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:55.099182 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1377/2755] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:54:54 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:54:55.103382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:55.113636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208603b6-5e7e-4f69-ad2d-73d67701b77b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:55.118778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:55.119368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:55.119641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:55.119906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:55.120596 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:55.122179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:55.132751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1379/2756] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:54:55.136572 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:55.146317 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833fdff-3e52-4faf-adb9-3d9d1c537e5c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:55.151168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662245f5-c1b4-438c-8b66-679c0888a525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:55.151756 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662245f5-c1b4-438c-8b66-679c0888a525 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:55.152035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662245f5-c1b4-438c-8b66-679c0888a525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:55.152344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662245f5-c1b4-438c-8b66-679c0888a525 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:55.152982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662245f5-c1b4-438c-8b66-679c0888a525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:55.154554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662245f5-c1b4-438c-8b66-679c0888a525 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:55.165221 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1378/2757] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:54:55.169206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:55.179779 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5f982-2f87-4bb7-8344-0b13c484c03a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:55.184451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:55.185094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:55.185375 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:55.185650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] PATH_INFO: `/v3/projects/50740a5d6e8942479c77e033fc124935/users/24cefb8650864ffdb4851ec1fdc5ef4e/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:55.201707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=50740a5d6e8942479c77e033fc124935, user_id=24cefb8650864ffdb4851ec1fdc5ef4e, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:55.204354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:55.293603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1380/2758] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:54:55 2022] PUT /identity/v3/projects/50740a5d6e8942479c77e033fc124935/users/24cefb8650864ffdb4851ec1fdc5ef4e/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:54:55.304543 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e360-7cb4-4ea1-b812-46e65c654015 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:55.304830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e360-7cb4-4ea1-b812-46e65c654015 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:55.305103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e360-7cb4-4ea1-b812-46e65c654015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:55.333184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c462e360-7cb4-4ea1-b812-46e65c654015 None None] MFA Rules not processed for user `24cefb8650864ffdb4851ec1fdc5ef4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:55.334405 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c462e360-7cb4-4ea1-b812-46e65c654015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:55.398845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1379/2759] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:54:55.409862 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:54:55.411627 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:55.468829 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda99bd5-98bf-4160-a067-7e41c2ceb3fb admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:54:55.472452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:54:55.473186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:55.473472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:55.473747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:55.482412 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:54:55.483837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:54:55.493141 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1381/2760] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:54:57.809016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:54:57.809508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:54:57.809607 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:54:57.828753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] MFA Rules not processed for user `2a6a637072b04667a3f21da9c57325c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:54:57.829950 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:54:57.842117 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:54:57.888418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1380/2761] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:55:01.312609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:01.314632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:01.378996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1012a-52b1-47a6-be87-9f834fad9302 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:01.382722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:01.383842 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:01.384119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:01.384480 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:01.386184 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:01.419094 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:01.451499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:01.452960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4595618-cf9a-403e-8817-b0d1cc7255d4 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:01.463044 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1382/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:55:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:55:07.901457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=17 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:08.275032 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:08.337423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5d5d-1764-40c8-a27c-29c810ad0c79 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:08.341873 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:08.342502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:08.342780 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:08.343056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] PATH_INFO: `/v3/users/24cefb8650864ffdb4851ec1fdc5ef4e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:08.344187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24cefb8650864ffdb4851ec1fdc5ef4e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:08.346102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:08.515583 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24cefb8650864ffdb4851ec1fdc5ef4e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:08.516299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24cefb8650864ffdb4851ec1fdc5ef4e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:08.516913 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24cefb8650864ffdb4851ec1fdc5ef4e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:08.608530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24cefb8650864ffdb4851ec1fdc5ef4e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:08.616457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24cefb8650864ffdb4851ec1fdc5ef4e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:08.635827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1381/2763] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:55:08 2022] DELETE /identity/v3/users/24cefb8650864ffdb4851ec1fdc5ef4e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:55:08.640237 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:08.640748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:08.640838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:08.661034 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:55:08.662215 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:08.725017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1383/2764] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:55:08.736374 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:08.736545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:08.794175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bdb79-7113-4355-9faa-f29bf946422d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:08.797985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab05ba-443a-47d4-a7d0-52792ef37f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:08.798587 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab05ba-443a-47d4-a7d0-52792ef37f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:08.798865 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab05ba-443a-47d4-a7d0-52792ef37f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:08.799136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab05ba-443a-47d4-a7d0-52792ef37f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:08.808716 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab05ba-443a-47d4-a7d0-52792ef37f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:08.810149 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab05ba-443a-47d4-a7d0-52792ef37f4e service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:08.820201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1382/2765] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:55:09.229655 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:09.231112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:09.243430 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4de78-2886-4493-9dab-7b3a39b83526 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:09.248668 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:09.249302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:09.249581 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:09.249855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] PATH_INFO: `/v3/projects/50740a5d6e8942479c77e033fc124935` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:09.250990 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50740a5d6e8942479c77e033fc124935)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:09.253031 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:09.359609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50740a5d6e8942479c77e033fc124935', 'request_id': 'req-78f2b7e2-f09a-4426-a61d-3a4fad31f017'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:09.432855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50740a5d6e8942479c77e033fc124935', 'request_id': 'req-78f2b7e2-f09a-4426-a61d-3a4fad31f017'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:09.438551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50740a5d6e8942479c77e033fc124935', 'request_id': 'req-78f2b7e2-f09a-4426-a61d-3a4fad31f017'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:09.449850 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] The token cache is being invalidate because project 50740a5d6e8942479c77e033fc124935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:55:09.450140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:09.450784 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:09.452310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1384/2766] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:55:09 2022] DELETE /identity/v3/projects/50740a5d6e8942479c77e033fc124935 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:55:18.815428 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:19.446682 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:27.442935 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:27.443411 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:27.451201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:27.463675 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] MFA Rules not processed for user `6e21a071d561423689653593b30cdaed`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:55:27.464922 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:27.520226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1383/2767] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:55:37.513511 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=25 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:41.490868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:41.493072 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:41.557347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2b7e2-f09a-4426-a61d-3a4fad31f017 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:41.561268 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:41.561891 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:41.562175 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:41.562447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:41.564443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:41.633536 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:41.635008 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:41.645531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1385/2768] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:55:48.817080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:48.818498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:48.890064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ee7aa-471d-4358-8cd9-8189df255b72 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:48.894941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:48.895544 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:48.895819 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:48.896086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:48.898007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:48.909459 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:48.911016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:48.920928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1384/2769] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:55:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:55:48.954878 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:48.956514 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:48.970966 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c36d44-b60d-4c6b-9ad9-0a7336ee5cb2 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:48.976670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:48.980477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:48.980477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:48.980477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] PATH_INFO: `/v3/users/2d34e301934743d5a457ae7fd979a53f` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:48.980477 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d34e301934743d5a457ae7fd979a53f)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:48.981314 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:49.177702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d34e301934743d5a457ae7fd979a53f'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.244699 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d34e301934743d5a457ae7fd979a53f'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.254569 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d34e301934743d5a457ae7fd979a53f'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.255271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d34e301934743d5a457ae7fd979a53f'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.258629 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d34e301934743d5a457ae7fd979a53f'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.275509 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1386/2770] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:55:48 2022] DELETE /identity/v3/users/2d34e301934743d5a457ae7fd979a53f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:55:49.548518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:49.559821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bf4a2-49e4-479a-b545-44417676ef3b service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:49.615628 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:49.616932 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:49.617209 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:49.617491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] PATH_INFO: `/v3/projects/6bc6eee4ac194763a27bb87522d47c6c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:49.618573 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc6eee4ac194763a27bb87522d47c6c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:49.620480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:49.748388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc6eee4ac194763a27bb87522d47c6c', 'request_id': 'req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.846888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc6eee4ac194763a27bb87522d47c6c', 'request_id': 'req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.853708 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc6eee4ac194763a27bb87522d47c6c', 'request_id': 'req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.864720 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] The token cache is being invalidate because project 6bc6eee4ac194763a27bb87522d47c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:55:49.865023 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.865632 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:49.867094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1385/2771] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:55:49 2022] DELETE /identity/v3/projects/6bc6eee4ac194763a27bb87522d47c6c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:55:49.871097 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:49.872603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:49.885937 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d537a-5eb2-4387-82eb-11270df05e85 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:49.904608 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:49.942542 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:49.943162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:49.943436 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:49.943704 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] PATH_INFO: `/v3/users/207052f6f3b746e3921e3c0cc2d902ff` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:49.944794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] RBAC: Authorizing `identity:delete_user(user_id=207052f6f3b746e3921e3c0cc2d902ff)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:49.946823 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:50.098737 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207052f6f3b746e3921e3c0cc2d902ff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.148431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207052f6f3b746e3921e3c0cc2d902ff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.155942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207052f6f3b746e3921e3c0cc2d902ff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.156636 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207052f6f3b746e3921e3c0cc2d902ff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.157320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207052f6f3b746e3921e3c0cc2d902ff'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.175227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1387/2772] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:55:49 2022] DELETE /identity/v3/users/207052f6f3b746e3921e3c0cc2d902ff => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:55:50.556443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:55:50.557877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:55:50.572432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984be17-bac8-40ef-a22c-c0a5cbb9f0da admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:55:50.597746 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:55:50.638244 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:55:50.638908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:55:50.639185 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:55:50.639449 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] PATH_INFO: `/v3/projects/35a25066d1b4476a91bd4777b4bffb63` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:55:50.640695 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a25066d1b4476a91bd4777b4bffb63)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:55:50.642577 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:55:50.751785 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a25066d1b4476a91bd4777b4bffb63', 'request_id': 'req-a30823f6-8073-461a-8eaa-15680e489d25'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.853542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a25066d1b4476a91bd4777b4bffb63', 'request_id': 'req-a30823f6-8073-461a-8eaa-15680e489d25'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.859773 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a25066d1b4476a91bd4777b4bffb63', 'request_id': 'req-a30823f6-8073-461a-8eaa-15680e489d25'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.871979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] The token cache is being invalidate because project 35a25066d1b4476a91bd4777b4bffb63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:55:50.874591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.874591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:55:50.874591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1386/2773] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:55:50 2022] DELETE /identity/v3/projects/35a25066d1b4476a91bd4777b4bffb63 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:56:00.171489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:00.868770 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:14.405491 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:14.411265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:14.464658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d344a-f839-4f2c-8502-94d07b45d62d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:14.512546 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:14.513147 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:14.513428 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:14.513700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:14.515429 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:14.586582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:14.588020 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:14.598965 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1388/2774] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:15.105659 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:15.107555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:15.132424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30823f6-8073-461a-8eaa-15680e489d25 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:15.140956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:15.141924 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:15.142370 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:15.142740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:15.147305 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:15.162908 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:15.165606 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:15.177508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1387/2775] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:15.250107 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:15.252046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:15.263025 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007624-57fb-4ba7-8c5a-25aeb109b045 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:15.267762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:15.270673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:15.270673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:15.270673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:15.271658 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:15.283002 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:15.284476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:15.293913 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1389/2776] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:17.793738 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:17.795150 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:17.853173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359368f-f7fb-460b-8e89-d953d5c30958 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:17.856943 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:17.860096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:17.860096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:17.860096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:17.860096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:17.873909 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:17.875350 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364bab3-a56e-42ec-bf91-d70dcfce5566 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:17.888964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1388/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:22.010210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:22.011722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:22.028490 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bf311-0a83-4871-844d-fbd5685ab755 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:22.034200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb8a6f42b6e482eb109be9d2f40e64f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:22.034739 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:22.035016 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:22.035289 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:22.037012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:22.049548 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:22.051152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:22.064344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1390/2778] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:23.286970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:23.287434 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:23.287534 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:23.337494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] MFA Rules not processed for user `438dc1eb842e4106b704a0a3c2a88fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:56:23.338701 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:23.401851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1389/2779] 10.222.0.57 () {58 vars in 967 bytes} [Wed Sep 7 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:56:23.442295 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:23.443698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:23.482307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=100 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:23.539816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445fa17-1c79-4366-adde-5c83334def62 service nova] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:23.544950 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256b9d949c8e4734a6bce965a107c060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:23.545553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:23.545828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:23.546099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:23.554043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:23.555626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:23.563411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1391/2780] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Sep 7 14:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 14:56:31.832538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:31.834026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:31.901792 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cae78-cced-4a80-99f5-ae471fb0257b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:31.906576 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:31.907200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:31.907475 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:31.907743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] PATH_INFO: `/v3/users/c18dbeea047a4505b2d5417dc54e8cb4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:31.912860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18dbeea047a4505b2d5417dc54e8cb4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:31.912860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:31.913673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:32.111988 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18dbeea047a4505b2d5417dc54e8cb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.112661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18dbeea047a4505b2d5417dc54e8cb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.113253 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18dbeea047a4505b2d5417dc54e8cb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.177920 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18dbeea047a4505b2d5417dc54e8cb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.185882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18dbeea047a4505b2d5417dc54e8cb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.203161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1390/2781] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:56:31 2022] DELETE /identity/v3/users/c18dbeea047a4505b2d5417dc54e8cb4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:32.445884 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:32.456972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd677a-215f-439d-82fa-9569caa1ca3d service swift] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:32.634928 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:32.635550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:32.635830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:32.636101 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] PATH_INFO: `/v3/projects/2710cdaf256342faae76fd5451ed432b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:32.637239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2710cdaf256342faae76fd5451ed432b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:32.639119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:32.722977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2710cdaf256342faae76fd5451ed432b', 'request_id': 'req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.775127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2710cdaf256342faae76fd5451ed432b', 'request_id': 'req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.781187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2710cdaf256342faae76fd5451ed432b', 'request_id': 'req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.792447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] The token cache is being invalidate because project 2710cdaf256342faae76fd5451ed432b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:56:32.792736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.793373 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:32.794879 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1392/2782] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:56:32 2022] DELETE /identity/v3/projects/2710cdaf256342faae76fd5451ed432b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:56:32.798991 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:32.800439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:32.813776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37fc76-9d51-4320-b106-57fed1c2ffda admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:32.870419 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:32.871126 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:32.871283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:32.871555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] PATH_INFO: `/v3/users/3bdb3970954a4f979f0c470ac7857254` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:32.872763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdb3970954a4f979f0c470ac7857254)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:32.874759 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:33.033768 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdb3970954a4f979f0c470ac7857254'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.034486 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdb3970954a4f979f0c470ac7857254'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.035101 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdb3970954a4f979f0c470ac7857254'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.103454 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdb3970954a4f979f0c470ac7857254'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.110596 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdb3970954a4f979f0c470ac7857254'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.124426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:33.128883 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1391/2783] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:56:32 2022] DELETE /identity/v3/users/3bdb3970954a4f979f0c470ac7857254 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:33.389120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:33.390609 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:33.408362 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c183ab1-17ae-4fb6-ba28-cca04c0eda9d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:33.438827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:33.466617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:33.467302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:33.467511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:33.467783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] PATH_INFO: `/v3/projects/e572a18003fb4d80982b0349bf8490ae` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:33.468925 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e572a18003fb4d80982b0349bf8490ae)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:33.470824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:33.566809 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e572a18003fb4d80982b0349bf8490ae', 'request_id': 'req-0fe25a58-85ed-4f22-b200-9297cb7e63c6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.626868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e572a18003fb4d80982b0349bf8490ae', 'request_id': 'req-0fe25a58-85ed-4f22-b200-9297cb7e63c6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.632744 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e572a18003fb4d80982b0349bf8490ae', 'request_id': 'req-0fe25a58-85ed-4f22-b200-9297cb7e63c6'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.643557 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] The token cache is being invalidate because project e572a18003fb4d80982b0349bf8490ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:56:33.643849 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.644495 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:33.645975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1393/2784] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:56:33 2022] DELETE /identity/v3/projects/e572a18003fb4d80982b0349bf8490ae => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:41.915802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:41.917398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:41.977894 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7a13-e15f-4554-abcb-6aa169207990 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:42.025322 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:42.025928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:42.026215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:42.026487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:42.028227 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:42.086764 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:42.088279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:42.097112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1392/2785] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:43.650176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:52.093120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=68 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:53.146009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:53.147443 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:53.204639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe25a58-85ed-4f22-b200-9297cb7e63c6 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:53.208169 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:53.208777 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:53.209057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:53.209329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:53.211649 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:53.272854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:53.275205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:53.284651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1394/2786] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:56:56.989891 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:56.990400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:57.005657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99f82c-dbdb-4aea-aed5-e18dbd34ddbb service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:57.010707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:57.014215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:57.014215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:57.014215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] PATH_INFO: `/v3/users/6e21a071d561423689653593b30cdaed` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:57.014215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e21a071d561423689653593b30cdaed)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:57.015242 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:57.192621 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e21a071d561423689653593b30cdaed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.193582 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e21a071d561423689653593b30cdaed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.194457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e21a071d561423689653593b30cdaed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.284241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e21a071d561423689653593b30cdaed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.296436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-19667fca-fdf3-407a-8236-2792b127573a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e21a071d561423689653593b30cdaed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.320713 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1393/2787] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:56:56 2022] DELETE /identity/v3/users/6e21a071d561423689653593b30cdaed => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:57.641346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:57.654740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9587c-1149-4af9-b704-bce37a404f3f service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:57.728254 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:57.728893 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:57.729170 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:57.729439 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] PATH_INFO: `/v3/projects/375694ab50a34237bf53010138421f1b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:57.730578 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=375694ab50a34237bf53010138421f1b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:57.732545 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:57.839911 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375694ab50a34237bf53010138421f1b', 'request_id': 'req-018af2d1-65ae-41da-8fde-014a6d34abbb'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.842680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:57.977912 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375694ab50a34237bf53010138421f1b', 'request_id': 'req-018af2d1-65ae-41da-8fde-014a6d34abbb'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.984423 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375694ab50a34237bf53010138421f1b', 'request_id': 'req-018af2d1-65ae-41da-8fde-014a6d34abbb'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.996666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] The token cache is being invalidate because project 375694ab50a34237bf53010138421f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:56:57.997059 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.997718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-018af2d1-65ae-41da-8fde-014a6d34abbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:56:57.999219 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1395/2788] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:56:57 2022] DELETE /identity/v3/projects/375694ab50a34237bf53010138421f1b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:58.004753 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04135383-9364-4980-9101-9396017b8a04 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.005384 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04135383-9364-4980-9101-9396017b8a04 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.005538 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04135383-9364-4980-9101-9396017b8a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.026554 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-04135383-9364-4980-9101-9396017b8a04 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:56:58.027788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-04135383-9364-4980-9101-9396017b8a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:58.093120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1394/2789] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:56:58.101227 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db830-e640-4763-af12-d6540bda3869 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.101518 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db830-e640-4763-af12-d6540bda3869 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.101794 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db830-e640-4763-af12-d6540bda3869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.122322 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-220db830-e640-4763-af12-d6540bda3869 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:56:58.123307 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-220db830-e640-4763-af12-d6540bda3869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:58.131342 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1396/2790] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:56:58.136376 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04135383-9364-4980-9101-9396017b8a04 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.146619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04135383-9364-4980-9101-9396017b8a04 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.152258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.153026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.153367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.153722 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.154431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:56:58.154813 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.156438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.170884 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1395/2791] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:56:58.175173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220db830-e640-4763-af12-d6540bda3869 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.184947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220db830-e640-4763-af12-d6540bda3869 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.189607 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.190193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.190471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.190736 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.191355 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.193302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.287472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1397/2792] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:56:58.292103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.302437 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27668e47-916f-40c6-92d2-722047ca65af admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.357365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.357962 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.358243 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.358518 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.359155 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.361099 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.369669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:58.446138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1396/2793] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:56:58.450191 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.460550 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2a7bd-2da7-4593-a9a1-160a6046b681 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.465438 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.466041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.466320 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.466589 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.467234 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.468868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.479123 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1398/2794] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:56:58.482979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.493566 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48883ee1-0cf8-472b-a2e9-312be34ba65e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.498201 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eeb19-9fd5-410c-a784-8b9c17a25c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.498799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eeb19-9fd5-410c-a784-8b9c17a25c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.499076 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eeb19-9fd5-410c-a784-8b9c17a25c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.499346 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eeb19-9fd5-410c-a784-8b9c17a25c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.499982 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4eeb19-9fd5-410c-a784-8b9c17a25c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.501608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4eeb19-9fd5-410c-a784-8b9c17a25c93 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.512532 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1397/2795] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:56:58.516631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.526602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cbcd-03ed-4a63-a244-0616520dbe5d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.531122 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.531749 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.532024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.532329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] PATH_INFO: `/v3/projects/588ef6f010c84e2f86dc35d7f5b70f6c/users/37af2fdff8ab4f0d9fa9dc6249b9aa0b/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.547262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] RBAC: Authorizing `identity:create_grant(project_id=588ef6f010c84e2f86dc35d7f5b70f6c, user_id=37af2fdff8ab4f0d9fa9dc6249b9aa0b, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.549918 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.618129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1399/2796] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:56:58 2022] PUT /identity/v3/projects/588ef6f010c84e2f86dc35d7f5b70f6c/users/37af2fdff8ab4f0d9fa9dc6249b9aa0b/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:58.636094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.636444 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.636787 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.666233 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] MFA Rules not processed for user `37af2fdff8ab4f0d9fa9dc6249b9aa0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:56:58.667732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:58.738810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1398/2797] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:56:58.743511 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.809156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef3f0-1399-4784-a78f-3e9580d3416a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:58.813982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:58.814592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:58.814865 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:58.815138 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:58.815751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:58.817669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:58.949024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1400/2798] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:56:58.953271 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:58.963799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de175b16-f112-4216-83cb-6f7bb5056b4e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.021634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.022321 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.022442 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.022699 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.023340 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.025541 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.127344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1399/2799] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:56:58 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:56:59.131543 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.142338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab1b04-a4bc-453d-8dce-658a4bc75001 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.147448 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.148055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.148371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.148641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.149538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.151162 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.161787 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1401/2800] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:56:59.166465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.176670 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14aa7f-a3eb-4633-85cb-24b98437a041 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.181998 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d003-6ea9-4f90-9ea5-099678d73c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.182599 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d003-6ea9-4f90-9ea5-099678d73c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.182876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d003-6ea9-4f90-9ea5-099678d73c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.183146 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d003-6ea9-4f90-9ea5-099678d73c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.183796 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d003-6ea9-4f90-9ea5-099678d73c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.185432 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d003-6ea9-4f90-9ea5-099678d73c6a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.197600 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1400/2801] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:56:59.204280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.215245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5fc4-1788-482f-bc18-0e944f061e8b admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.220164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.220834 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.221112 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.221383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] PATH_INFO: `/v3/projects/5ce8d751a74f4d3d8634d4b738b84d7b/users/bf56d2a58f9e48039fdec9c776f6e926/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.236374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce8d751a74f4d3d8634d4b738b84d7b, user_id=bf56d2a58f9e48039fdec9c776f6e926, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.239050 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.310707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1402/2802] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:56:59 2022] PUT /identity/v3/projects/5ce8d751a74f4d3d8634d4b738b84d7b/users/bf56d2a58f9e48039fdec9c776f6e926/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:59.322929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.323213 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.323488 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.351889 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] MFA Rules not processed for user `bf56d2a58f9e48039fdec9c776f6e926`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:56:59.359083 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:56:59.389786 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:59.423774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1401/2803] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:56:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:56:59.428567 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.449017 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:56:59.490052 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5033fd-ebd5-4b76-b892-9e0dc38a9d00 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.495261 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.495856 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.496413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.496413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.497156 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.499066 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.625631 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1403/2804] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:56:59 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:56:59.629905 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.643772 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec740ea5-014b-44d4-b276-4c145bed46bc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.701163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.701810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.702094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.702367 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.703006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.705113 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.787047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1402/2805] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:56:59 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:56:59.791783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.802645 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ace8b-9933-4cd9-8f6b-b51375f4749f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.808461 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.809089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.809325 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.809592 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.810216 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.811868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.822738 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1404/2806] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:56:59.828906 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.839408 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdbe7-4cae-4886-b641-4c9efd9df37b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:56:59.844230 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:56:59.844885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:56:59.845165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:56:59.845457 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] PATH_INFO: `/v3/projects/0473383544b440e681f02d2618cc7669/users/64780e7328794e9582436e18dc41be07/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:56:59.860858 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0473383544b440e681f02d2618cc7669, user_id=64780e7328794e9582436e18dc41be07, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:56:59.863557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:56:59.953700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1403/2807] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:56:59 2022] PUT /identity/v3/projects/0473383544b440e681f02d2618cc7669/users/64780e7328794e9582436e18dc41be07/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:56:59.954564 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:56:59.965924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87d404-ec13-4211-909d-4d9d2e450f0e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:00.045895 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19dae-4566-4ee7-ab36-62eb47d2b319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:00.046515 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19dae-4566-4ee7-ab36-62eb47d2b319 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:00.046805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19dae-4566-4ee7-ab36-62eb47d2b319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:00.047081 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19dae-4566-4ee7-ab36-62eb47d2b319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:00.047726 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d19dae-4566-4ee7-ab36-62eb47d2b319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:00.049366 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d19dae-4566-4ee7-ab36-62eb47d2b319 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:00.062110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1405/2808] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:00.064767 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:00.075090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974247-9c47-4132-9003-9a99bbb5b43e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:00.079653 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:00.080338 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:00.080617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:00.080887 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] PATH_INFO: `/v3/projects/0473383544b440e681f02d2618cc7669/users/64780e7328794e9582436e18dc41be07/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:00.082791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0473383544b440e681f02d2618cc7669, user_id=64780e7328794e9582436e18dc41be07, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:00.086025 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:00.096048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1404/2809] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:00 2022] PUT /identity/v3/projects/0473383544b440e681f02d2618cc7669/users/64780e7328794e9582436e18dc41be07/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:00.106814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:00.107100 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:00.107374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:00.135136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] MFA Rules not processed for user `64780e7328794e9582436e18dc41be07`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:00.136405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:00.206078 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1406/2810] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:00.217335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:00.219036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:00.277809 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5673dd5-1e19-47c1-9bc4-be043ead59eb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:00.281553 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:00.282161 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:00.282436 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:00.282707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:00.292714 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:00.294303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:00.306917 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1405/2811] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:01.405909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:01.407341 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:01.420324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9a42b-4b63-4bc0-80b7-b9c181673f5a None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:01.423875 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:01.424512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:01.424793 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:01.425062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:01.435275 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:01.482486 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:01.483907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:01.493593 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1407/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:03.316168 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:03.325623 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:03.362068 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:03.386402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b601b-1243-4a1a-870d-ff131c53a535 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:03.390041 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:03.390645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:03.390929 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:03.391280 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:03.393127 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:03.457682 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:03.459123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:03.469386 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1406/2813] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:57:06.578183 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:06.587389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882fdb-bcf4-4cd7-9492-277f9937be8e service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:06.591434 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:06.593741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:06.593741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:06.593741 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:06.684923 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:06.686583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:06.697775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1408/2814] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:10.589409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:10.590885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:10.603928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bef37-a5f3-4178-bce4-7f337a68bfd0 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:10.608810 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:10.609461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:10.609807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:10.610093 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] PATH_INFO: `/v3/users/2a6a637072b04667a3f21da9c57325c0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:10.611447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6a637072b04667a3f21da9c57325c0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:10.613467 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:10.828708 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:10.840067 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13595dd7-7b4e-4ad9-a76c-88edc55e3234 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:10.844691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:10.845371 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:10.845646 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:10.846128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] PATH_INFO: `/v3/users/37af2fdff8ab4f0d9fa9dc6249b9aa0b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:10.847250 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37af2fdff8ab4f0d9fa9dc6249b9aa0b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:10.849200 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:10.850086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6a637072b04667a3f21da9c57325c0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:10.850774 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6a637072b04667a3f21da9c57325c0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:10.851898 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6a637072b04667a3f21da9c57325c0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:10.949443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6a637072b04667a3f21da9c57325c0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:10.957189 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ab4b4272-2ef1-4960-93d1-dd0eed2db956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6a637072b04667a3f21da9c57325c0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:10.981324 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1407/2815] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:10 2022] DELETE /identity/v3/users/2a6a637072b04667a3f21da9c57325c0 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:11.063586 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37af2fdff8ab4f0d9fa9dc6249b9aa0b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.143243 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37af2fdff8ab4f0d9fa9dc6249b9aa0b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.151080 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37af2fdff8ab4f0d9fa9dc6249b9aa0b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.151758 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37af2fdff8ab4f0d9fa9dc6249b9aa0b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.152385 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37af2fdff8ab4f0d9fa9dc6249b9aa0b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.171306 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1409/2816] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:10 2022] DELETE /identity/v3/users/37af2fdff8ab4f0d9fa9dc6249b9aa0b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:11.175799 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:11.176107 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:11.176409 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:11.196593 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:11.197854 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:11.232626 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:11.234782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:11.252440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:11.271114 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1408/2817] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:11.282798 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:11.283589 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:11.288807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:11.340110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce199489-8745-43de-8bdc-34b47962d06d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:11.342781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:11.343508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:11.343877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:11.344245 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] PATH_INFO: `/v3/projects/5c7b26a4966c4c309d058d1d19d5d553` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:11.345625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7b26a4966c4c309d058d1d19d5d553)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:11.347632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:11.354172 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37373ad8-4101-4bd2-9264-b875aa8a7120 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:11.358141 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:11.358872 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:11.359162 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:11.359506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:11.372947 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:11.375542 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:11.386290 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1409/2818] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:11.453249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7b26a4966c4c309d058d1d19d5d553', 'request_id': 'req-b68f21e6-e602-4498-a944-ce319fe2d921'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.532251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7b26a4966c4c309d058d1d19d5d553', 'request_id': 'req-b68f21e6-e602-4498-a944-ce319fe2d921'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.538192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7b26a4966c4c309d058d1d19d5d553', 'request_id': 'req-b68f21e6-e602-4498-a944-ce319fe2d921'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.549181 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] The token cache is being invalidate because project 5c7b26a4966c4c309d058d1d19d5d553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:11.549463 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.550127 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.551854 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1410/2819] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:11 2022] DELETE /identity/v3/projects/5c7b26a4966c4c309d058d1d19d5d553 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:11.556410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:11.566602 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f246-f55d-424f-8c1f-0a220db7173f service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:11.633105 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:11.633868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:11.634125 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:11.634278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] PATH_INFO: `/v3/users/725b741992fe4c39939e2f29a72a3ab1` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:11.635517 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=725b741992fe4c39939e2f29a72a3ab1)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:11.637520 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:11.819376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:11.820930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:11.834648 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f21e6-e602-4498-a944-ce319fe2d921 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:11.839462 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:11.840027 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:11.840444 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:11.841153 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] PATH_INFO: `/v3/projects/588ef6f010c84e2f86dc35d7f5b70f6c` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:11.842042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588ef6f010c84e2f86dc35d7f5b70f6c)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:11.844010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:11.865139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725b741992fe4c39939e2f29a72a3ab1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.865794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725b741992fe4c39939e2f29a72a3ab1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.867050 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725b741992fe4c39939e2f29a72a3ab1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.978119 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725b741992fe4c39939e2f29a72a3ab1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.986610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725b741992fe4c39939e2f29a72a3ab1'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:11.996977 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588ef6f010c84e2f86dc35d7f5b70f6c', 'request_id': 'req-f79f5bdf-888e-4019-a07b-f896db0179e2'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.005857 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1410/2820] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:11 2022] DELETE /identity/v3/users/725b741992fe4c39939e2f29a72a3ab1 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:12.081009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588ef6f010c84e2f86dc35d7f5b70f6c', 'request_id': 'req-f79f5bdf-888e-4019-a07b-f896db0179e2'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.088092 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588ef6f010c84e2f86dc35d7f5b70f6c', 'request_id': 'req-f79f5bdf-888e-4019-a07b-f896db0179e2'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.099762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] The token cache is being invalidate because project 588ef6f010c84e2f86dc35d7f5b70f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:12.099978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.101722 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.103318 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1411/2821] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:11 2022] DELETE /identity/v3/projects/588ef6f010c84e2f86dc35d7f5b70f6c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:12.107458 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:12.108927 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:12.127423 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:12.174951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d8e10-abdd-4d2f-9605-a4845f436f0a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:12.179679 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:12.180348 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:12.180629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:12.180900 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] PATH_INFO: `/v3/users/bf56d2a58f9e48039fdec9c776f6e926` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:12.182010 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf56d2a58f9e48039fdec9c776f6e926)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:12.183958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:12.299474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:12.300946 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:12.344374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:12.369176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf56d2a58f9e48039fdec9c776f6e926'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.369821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf56d2a58f9e48039fdec9c776f6e926'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.369956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f5bdf-888e-4019-a07b-f896db0179e2 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:12.370555 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf56d2a58f9e48039fdec9c776f6e926'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.374508 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:12.375139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:12.375422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:12.375698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] PATH_INFO: `/v3/projects/03f993468f1842309b2f338ed903d1d5` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:12.376855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f993468f1842309b2f338ed903d1d5)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:12.378782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:12.469397 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf56d2a58f9e48039fdec9c776f6e926'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.477440 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf56d2a58f9e48039fdec9c776f6e926'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.487058 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f993468f1842309b2f338ed903d1d5', 'request_id': 'req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.495801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1411/2822] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:12 2022] DELETE /identity/v3/users/bf56d2a58f9e48039fdec9c776f6e926 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:12.566285 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f993468f1842309b2f338ed903d1d5', 'request_id': 'req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.572605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f993468f1842309b2f338ed903d1d5', 'request_id': 'req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.584999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] The token cache is being invalidate because project 03f993468f1842309b2f338ed903d1d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:12.585271 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.585921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:12.587468 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1412/2823] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:12 2022] DELETE /identity/v3/projects/03f993468f1842309b2f338ed903d1d5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:12.910987 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:12.912489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:12.980504 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c6e4b-5600-460d-9b1b-33299a18f38c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:12.985187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:12.985823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:12.986102 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:12.986372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] PATH_INFO: `/v3/projects/5ce8d751a74f4d3d8634d4b738b84d7b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:12.987489 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce8d751a74f4d3d8634d4b738b84d7b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:12.989430 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:13.080904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:13.089056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce8d751a74f4d3d8634d4b738b84d7b', 'request_id': 'req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.171815 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce8d751a74f4d3d8634d4b738b84d7b', 'request_id': 'req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.177806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce8d751a74f4d3d8634d4b738b84d7b', 'request_id': 'req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.189918 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] The token cache is being invalidate because project 5ce8d751a74f4d3d8634d4b738b84d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:13.190205 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.190838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.192364 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1412/2824] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:12 2022] DELETE /identity/v3/projects/5ce8d751a74f4d3d8634d4b738b84d7b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:13.197083 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:13.198517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:13.261927 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f7ceb-cf8d-4ca6-82df-07c60f0f4078 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:13.317354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:13.317979 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:13.318260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:13.318534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] PATH_INFO: `/v3/users/64780e7328794e9582436e18dc41be07` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:13.319653 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] RBAC: Authorizing `identity:delete_user(user_id=64780e7328794e9582436e18dc41be07)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:13.321603 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:13.334573 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:13.528431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64780e7328794e9582436e18dc41be07'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.614870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64780e7328794e9582436e18dc41be07'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.622055 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64780e7328794e9582436e18dc41be07'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.622728 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64780e7328794e9582436e18dc41be07'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.623359 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-e11469ba-f7b6-4f1f-8e00-bace975183db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64780e7328794e9582436e18dc41be07'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:13.640772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1413/2825] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:13 2022] DELETE /identity/v3/users/64780e7328794e9582436e18dc41be07 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:14.107762 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.116481 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:14.123215 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac442a-926d-4e5f-9476-b5c8ac0d8850 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.181824 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.182461 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.182742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.183017 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] PATH_INFO: `/v3/projects/0473383544b440e681f02d2618cc7669` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.184135 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0473383544b440e681f02d2618cc7669)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.186129 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.289821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0473383544b440e681f02d2618cc7669', 'request_id': 'req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:14.346385 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0473383544b440e681f02d2618cc7669', 'request_id': 'req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:14.352175 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0473383544b440e681f02d2618cc7669', 'request_id': 'req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:14.363079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] The token cache is being invalidate because project 0473383544b440e681f02d2618cc7669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:14.363368 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:14.364003 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6cdfca1c-b11a-4e27-b257-1a2b710d9d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:14.365528 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1413/2826] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:14 2022] DELETE /identity/v3/projects/0473383544b440e681f02d2618cc7669 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:14.372010 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.372521 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.372635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.392433 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:14.393642 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:14.452048 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1414/2827] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:14.461917 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.462200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.462470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.481823 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:14.483006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:14.490702 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1414/2828] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:14.495370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.504860 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4054f9-3b45-4468-bade-5f785d5ea2e3 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.509994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.510582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.510861 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.511129 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.511762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:57:14.512062 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.513715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.525186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1415/2829] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:57:14.529184 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.538251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784794bb-4d91-4f93-818b-98a19885b1df None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.542816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.543399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.543673 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.543941 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.544605 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.546480 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.627734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1415/2830] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:14 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:57:14.631717 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.642294 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b37fbd-0934-48cf-966e-b3476a3f2db5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.700651 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.701246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.701520 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.701813 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.702408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.704323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.780617 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1416/2831] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:14 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:14.784758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.795691 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b33c-cba8-465c-9cbf-68ed63587e78 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.800468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.801775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.801775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.801775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.802220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.804426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.814286 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1416/2832] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:57:14.818176 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.827883 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ac72b-7015-4219-9a75-2b327e6557ce admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.832549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662c07a-595e-453a-a5cf-c2c5bbdcd1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.833136 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662c07a-595e-453a-a5cf-c2c5bbdcd1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.833414 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662c07a-595e-453a-a5cf-c2c5bbdcd1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.833677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662c07a-595e-453a-a5cf-c2c5bbdcd1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.834303 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662c07a-595e-453a-a5cf-c2c5bbdcd1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.835887 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662c07a-595e-453a-a5cf-c2c5bbdcd1c6 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.837960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:14.846612 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1417/2833] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:14.851277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:14.861661 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cc23-ebc0-40cc-a165-9989f64ec5c9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:14.867007 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:14.867640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:14.867921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:14.868220 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] PATH_INFO: `/v3/projects/dec5e02866a541068dac15b7c55efb20/users/01599ffaaa0c47b3954377fcfdcdfa0d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:14.882121 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:14.883650 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec5e02866a541068dac15b7c55efb20, user_id=01599ffaaa0c47b3954377fcfdcdfa0d, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:14.886314 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:14.993656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1417/2834] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:14 2022] PUT /identity/v3/projects/dec5e02866a541068dac15b7c55efb20/users/01599ffaaa0c47b3954377fcfdcdfa0d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:15.005028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.005333 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.005600 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.032697 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] MFA Rules not processed for user `01599ffaaa0c47b3954377fcfdcdfa0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:15.033926 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:15.096447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1418/2835] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:15.101530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.163870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255827a-e4bd-4e80-8241-19e34b46f927 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.168580 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.169186 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.169464 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.169736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.170365 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.172293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.244629 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1418/2836] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:15 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:15.248698 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.259406 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b818f-bce8-4b79-b3f2-261d02ec29f1 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.315782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.316420 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.316702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.316973 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.317606 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.319544 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.395501 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1419/2837] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:15 2022] POST /identity/v3/users => generated 297 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:15.400196 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.410985 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bab39-6109-4587-b15c-d0cc5bedeffd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.420732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.420732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.420732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.420732 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.421199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.421199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.431079 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1419/2838] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:57:15.435338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.445870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323382e-ba93-498a-91ff-54c86abcecb1 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.450354 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.450994 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.451265 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.451528 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] PATH_INFO: `/v3/projects/311ef6db6a8247cf92f6dc50d806b3da/users/52dadab2c01846f7b771e869a472eb5b/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.467441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311ef6db6a8247cf92f6dc50d806b3da, user_id=52dadab2c01846f7b771e869a472eb5b, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.470228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.571759 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1420/2839] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:15 2022] PUT /identity/v3/projects/311ef6db6a8247cf92f6dc50d806b3da/users/52dadab2c01846f7b771e869a472eb5b/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:15.575948 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.586740 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c58f4c-6ede-47dc-8e1a-e0562a8455ac admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.642851 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d743b23-dc23-47b2-9855-c545c5e4f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.643456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d743b23-dc23-47b2-9855-c545c5e4f33c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.643734 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d743b23-dc23-47b2-9855-c545c5e4f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.644006 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d743b23-dc23-47b2-9855-c545c5e4f33c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.644677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d743b23-dc23-47b2-9855-c545c5e4f33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.646260 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d743b23-dc23-47b2-9855-c545c5e4f33c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.656729 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1420/2840] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:15.660783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.671003 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f20ad-0d88-4701-8f9f-aec0b9477c48 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.675556 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.676229 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.676510 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.676782 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] PATH_INFO: `/v3/projects/311ef6db6a8247cf92f6dc50d806b3da/users/52dadab2c01846f7b771e869a472eb5b/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.678803 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=311ef6db6a8247cf92f6dc50d806b3da, user_id=52dadab2c01846f7b771e869a472eb5b, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.681469 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.691889 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1421/2841] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:15 2022] PUT /identity/v3/projects/311ef6db6a8247cf92f6dc50d806b3da/users/52dadab2c01846f7b771e869a472eb5b/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:15.702575 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.702861 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.703138 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.731743 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] MFA Rules not processed for user `52dadab2c01846f7b771e869a472eb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:15.733035 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:15.780164 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:15.799940 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1421/2842] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:15.811747 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:15.813447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:15.856707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:15.871192 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5173931-56ca-450a-be2a-a7cc6201b50a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:15.874756 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:15.875364 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:15.875638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:15.875906 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:15.886035 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:15.887474 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:15.896941 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1422/2843] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:15.999939 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:16.010251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16726376-0907-43c7-80ac-a05474ed53bb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:16.013874 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dd87b-e91d-42b0-a242-ed303e55bf4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:16.014468 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dd87b-e91d-42b0-a242-ed303e55bf4b service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:16.014747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dd87b-e91d-42b0-a242-ed303e55bf4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:16.015016 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dd87b-e91d-42b0-a242-ed303e55bf4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:16.070563 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dd87b-e91d-42b0-a242-ed303e55bf4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:16.071978 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dd87b-e91d-42b0-a242-ed303e55bf4b service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:16.080750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1422/2844] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:19.177999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:19.239991 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6e16-acb9-46e1-a37d-634f52a3a6f5 service cinder] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:19.245154 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:19.245778 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:19.246051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:19.246319 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] PATH_INFO: `/v3/users/01599ffaaa0c47b3954377fcfdcdfa0d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:19.247415 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01599ffaaa0c47b3954377fcfdcdfa0d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:19.249378 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:19.402177 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01599ffaaa0c47b3954377fcfdcdfa0d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:19.474816 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01599ffaaa0c47b3954377fcfdcdfa0d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:19.482499 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01599ffaaa0c47b3954377fcfdcdfa0d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:19.483145 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01599ffaaa0c47b3954377fcfdcdfa0d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:19.483804 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01599ffaaa0c47b3954377fcfdcdfa0d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:19.501458 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1423/2845] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:19 2022] DELETE /identity/v3/users/01599ffaaa0c47b3954377fcfdcdfa0d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:19.505951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e426c9-6764-4b66-95be-523716236e40 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:19.507159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e426c9-6764-4b66-95be-523716236e40 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:19.507159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e426c9-6764-4b66-95be-523716236e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:19.525921 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-24e426c9-6764-4b66-95be-523716236e40 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:19.527108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-24e426c9-6764-4b66-95be-523716236e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:19.585439 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1423/2846] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:19.595258 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:19.596709 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:19.652666 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdfbcf-fbe0-42a3-a79d-599b12e02ad7 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:19.656239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:19.656828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:19.657105 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:19.657374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:19.666691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:19.668128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:19.677308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1424/2847] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:20.074123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e426c9-6764-4b66-95be-523716236e40 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:20.075687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-24e426c9-6764-4b66-95be-523716236e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:20.088398 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e426c9-6764-4b66-95be-523716236e40 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:20.093136 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:20.097658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:20.097658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:20.097658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] PATH_INFO: `/v3/projects/dec5e02866a541068dac15b7c55efb20` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:20.097658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec5e02866a541068dac15b7c55efb20)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:20.097658 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:20.212448 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec5e02866a541068dac15b7c55efb20', 'request_id': 'req-89fbfa6c-7a35-40c4-a955-e71b349b7600'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.280570 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec5e02866a541068dac15b7c55efb20', 'request_id': 'req-89fbfa6c-7a35-40c4-a955-e71b349b7600'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.286614 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec5e02866a541068dac15b7c55efb20', 'request_id': 'req-89fbfa6c-7a35-40c4-a955-e71b349b7600'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.298167 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] The token cache is being invalidate because project dec5e02866a541068dac15b7c55efb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:20.298456 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.299089 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.300618 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1424/2848] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:20 2022] DELETE /identity/v3/projects/dec5e02866a541068dac15b7c55efb20 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:20.305179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:20.311140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:20.314909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98401cf1-e7c6-4231-85f7-630243feaf4b service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:20.371844 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:20.372431 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:20.372713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:20.372984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] PATH_INFO: `/v3/users/52dadab2c01846f7b771e869a472eb5b` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:20.374111 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52dadab2c01846f7b771e869a472eb5b)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:20.376071 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:20.579605 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52dadab2c01846f7b771e869a472eb5b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.657762 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52dadab2c01846f7b771e869a472eb5b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.665270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52dadab2c01846f7b771e869a472eb5b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.665947 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52dadab2c01846f7b771e869a472eb5b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.666538 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-255bfed2-63d5-4ae9-975e-f0396aefdea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52dadab2c01846f7b771e869a472eb5b'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:20.684287 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1425/2849] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:20 2022] DELETE /identity/v3/users/52dadab2c01846f7b771e869a472eb5b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:21.070615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:21.072173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:21.085816 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbfa6c-7a35-40c4-a955-e71b349b7600 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:21.090669 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:21.143399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:21.144066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.144375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.144649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] PATH_INFO: `/v3/projects/311ef6db6a8247cf92f6dc50d806b3da` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.145769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] RBAC: Authorizing `identity:delete_project(project_id=311ef6db6a8247cf92f6dc50d806b3da)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:21.148225 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:21.257387 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311ef6db6a8247cf92f6dc50d806b3da', 'request_id': 'req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:21.343951 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311ef6db6a8247cf92f6dc50d806b3da', 'request_id': 'req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:21.353130 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311ef6db6a8247cf92f6dc50d806b3da', 'request_id': 'req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:21.365048 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] The token cache is being invalidate because project 311ef6db6a8247cf92f6dc50d806b3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:21.365418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:21.366124 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c89b45f5-51ed-4b4e-92c7-33b7c8c4c15d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:21.367616 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1425/2850] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:21 2022] DELETE /identity/v3/projects/311ef6db6a8247cf92f6dc50d806b3da => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:21.372707 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.373197 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.373337 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.392639 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:21.393939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:21.453110 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1426/2851] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:21.513356 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.513690 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.513997 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.533821 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:21.535026 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:21.543291 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1426/2852] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:21.547487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:21.557710 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c3430-1318-495e-b076-59b247e1f9b8 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:21.562220 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:21.562810 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.563088 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.563356 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.564021 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:57:21.564381 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:21.565984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:21.577207 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1427/2853] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:57:21.581208 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:21.590335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb3b9a-64ff-4dab-b7f6-e19dba9a9295 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:21.594901 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:21.595491 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.595769 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.596040 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.596704 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:21.598579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:21.754064 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1427/2854] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:21 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:21.762279 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:21.773471 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eefc1-9d9c-4afa-90e4-59a34a4ad001 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:21.781370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:21.830273 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:21.830924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.831203 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.831476 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.832098 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:21.834051 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:21.975326 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1428/2855] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:21 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:21.979539 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:21.990276 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2582b-7aff-4d6a-a84e-e47901d3a062 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:21.996061 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:21.996765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:21.997047 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:21.997316 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:21.998108 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:21.999811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.010551 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1428/2856] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:22.014718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.024900 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83829f-d1bd-4771-8994-30709294ea1c admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.029397 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d531-84d8-4a47-9aa2-59544d017995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.029985 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d531-84d8-4a47-9aa2-59544d017995 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.030262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d531-84d8-4a47-9aa2-59544d017995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.030534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d531-84d8-4a47-9aa2-59544d017995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.031168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955d531-84d8-4a47-9aa2-59544d017995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.032791 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955d531-84d8-4a47-9aa2-59544d017995 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.043099 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1429/2857] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:22.047270 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.057942 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051897b-0ef9-425d-b14b-237b5d7725a2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.062568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.063251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.063530 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.063807 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] PATH_INFO: `/v3/projects/9ee1e3bcd85742059bf3b4964a1527e7/users/ded735798f34434f90acd27065676a27/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.078878 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee1e3bcd85742059bf3b4964a1527e7, user_id=ded735798f34434f90acd27065676a27, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.081574 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.155091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1429/2858] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:22 2022] PUT /identity/v3/projects/9ee1e3bcd85742059bf3b4964a1527e7/users/ded735798f34434f90acd27065676a27/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:22.165859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.166140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.166405 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.193623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] MFA Rules not processed for user `ded735798f34434f90acd27065676a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:22.195715 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:22.257939 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1430/2859] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:22.262634 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.305247 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:22.327485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12028d85-ce9c-4ee3-a96e-b1d9c3909a49 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.332254 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.332860 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.333140 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.333413 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.334086 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.335975 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.453112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1430/2860] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:22 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 14:57:22.457262 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.467792 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f49fd-bf76-4191-ba1e-b55ad8a25978 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.524009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.524909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.525187 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.525456 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.526089 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.528014 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.640185 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1431/2861] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:22 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:22.644094 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.655354 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09c827-a9cb-48e8-b283-8d9e153346a4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.660053 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.660680 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.660960 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.661231 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.661868 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.664228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.674694 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1431/2862] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:22.678690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.688466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34933-e3ce-4a3e-a5d1-55a612eec77a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.692999 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.693641 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.693921 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.694193 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] PATH_INFO: `/v3/projects/b18670978b6145bcb80ac9c9007b1e02/users/c779951998324328a68831a29bc6e8a2/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.697276 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:22.708700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18670978b6145bcb80ac9c9007b1e02, user_id=c779951998324328a68831a29bc6e8a2, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.711855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.777774 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1432/2863] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:22 2022] PUT /identity/v3/projects/b18670978b6145bcb80ac9c9007b1e02/users/c779951998324328a68831a29bc6e8a2/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:22.781686 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.792953 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca902a5-871b-42e9-84df-68ae28aa632b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.849219 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcf658-1498-4f80-82c8-48303bf32491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.849830 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf658-1498-4f80-82c8-48303bf32491 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.850104 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf658-1498-4f80-82c8-48303bf32491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.851075 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf658-1498-4f80-82c8-48303bf32491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.851719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcf658-1498-4f80-82c8-48303bf32491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.853303 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcf658-1498-4f80-82c8-48303bf32491 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.863166 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1432/2864] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:22.867465 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:22.877659 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd748e9-6a9f-4076-8079-d203e7a6612d admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:22.882293 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:22.882930 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.883205 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.883472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] PATH_INFO: `/v3/projects/b18670978b6145bcb80ac9c9007b1e02/users/c779951998324328a68831a29bc6e8a2/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.885387 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18670978b6145bcb80ac9c9007b1e02, user_id=c779951998324328a68831a29bc6e8a2, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:22.888032 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:22.898689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1433/2865] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:22 2022] PUT /identity/v3/projects/b18670978b6145bcb80ac9c9007b1e02/users/c779951998324328a68831a29bc6e8a2/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:22.910279 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:22.910569 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:22.910964 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:22.939402 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] MFA Rules not processed for user `c779951998324328a68831a29bc6e8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:22.940657 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:23.010676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1433/2866] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:23.023316 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:23.025344 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:23.081119 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b0135-b051-417a-acff-954993d86cc3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:23.084960 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8962607-44fa-4909-b3e7-062ab6596d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:23.085551 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962607-44fa-4909-b3e7-062ab6596d7e service cinder] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:23.085828 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962607-44fa-4909-b3e7-062ab6596d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:23.086095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962607-44fa-4909-b3e7-062ab6596d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:23.095173 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8962607-44fa-4909-b3e7-062ab6596d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:23.096638 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8962607-44fa-4909-b3e7-062ab6596d7e service cinder] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:23.106299 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1434/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:57:23.239498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:23.272042 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:23.300383 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ea06d-eec1-46e9-8444-df92223a9d26 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:23.305485 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:23.306096 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:23.306372 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:23.306648 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] PATH_INFO: `/v3/users/ded735798f34434f90acd27065676a27` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:23.307750 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded735798f34434f90acd27065676a27)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:23.309687 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:23.488588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded735798f34434f90acd27065676a27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:23.489238 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded735798f34434f90acd27065676a27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:23.489841 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded735798f34434f90acd27065676a27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:23.555015 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded735798f34434f90acd27065676a27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:23.562192 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded735798f34434f90acd27065676a27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:23.580115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1434/2868] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:23 2022] DELETE /identity/v3/users/ded735798f34434f90acd27065676a27 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:23.584691 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:23.585217 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:23.585368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:23.603877 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:23.605120 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:23.665916 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1435/2869] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:23.675752 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:23.677226 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:23.735640 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2735-5f3d-4062-b762-d82f0bd91585 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:23.739237 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:23.739846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:23.740123 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:23.740426 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:23.750248 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:23.751736 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:23.761112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1435/2870] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:24.159210 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:24.160650 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:24.174324 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf4bcc-d9c1-4204-acfe-061aa5d766b5 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:24.179209 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:24.180503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:24.180503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:24.180503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] PATH_INFO: `/v3/projects/9ee1e3bcd85742059bf3b4964a1527e7` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:24.181665 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee1e3bcd85742059bf3b4964a1527e7)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:24.183571 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:24.271716 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee1e3bcd85742059bf3b4964a1527e7', 'request_id': 'req-97b25bb5-53d1-401b-9d15-4a5ab55021bd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.360452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee1e3bcd85742059bf3b4964a1527e7', 'request_id': 'req-97b25bb5-53d1-401b-9d15-4a5ab55021bd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.366611 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee1e3bcd85742059bf3b4964a1527e7', 'request_id': 'req-97b25bb5-53d1-401b-9d15-4a5ab55021bd'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.373837 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:24.379272 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] The token cache is being invalidate because project 9ee1e3bcd85742059bf3b4964a1527e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:24.379555 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.381972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.381972 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1436/2871] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:24 2022] DELETE /identity/v3/projects/9ee1e3bcd85742059bf3b4964a1527e7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:24.385771 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:24.395592 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31419819-0e79-4e70-abab-cf756b610249 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:24.443882 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:24.452961 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:24.453590 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:24.453870 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:24.454169 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] PATH_INFO: `/v3/users/c779951998324328a68831a29bc6e8a2` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:24.455281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c779951998324328a68831a29bc6e8a2)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:24.457358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:24.617399 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c779951998324328a68831a29bc6e8a2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.618272 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c779951998324328a68831a29bc6e8a2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.618980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c779951998324328a68831a29bc6e8a2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.688846 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c779951998324328a68831a29bc6e8a2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.696265 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-066f4304-8c78-4c9a-8797-e289f48f1cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c779951998324328a68831a29bc6e8a2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:24.714335 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1436/2872] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:24 2022] DELETE /identity/v3/users/c779951998324328a68831a29bc6e8a2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:25.115725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.115725 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:25.130043 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b25bb5-53d1-401b-9d15-4a5ab55021bd admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.189186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.189815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.190095 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.190372 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] PATH_INFO: `/v3/projects/b18670978b6145bcb80ac9c9007b1e02` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.191587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18670978b6145bcb80ac9c9007b1e02)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.193531 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:25.312830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18670978b6145bcb80ac9c9007b1e02', 'request_id': 'req-454a38b4-7eed-4dd3-84af-60e12423303c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:25.394399 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18670978b6145bcb80ac9c9007b1e02', 'request_id': 'req-454a38b4-7eed-4dd3-84af-60e12423303c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:25.400374 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18670978b6145bcb80ac9c9007b1e02', 'request_id': 'req-454a38b4-7eed-4dd3-84af-60e12423303c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:25.411859 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] The token cache is being invalidate because project b18670978b6145bcb80ac9c9007b1e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81804) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:25.412206 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:25.422873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-454a38b4-7eed-4dd3-84af-60e12423303c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:25.422873 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1437/2873] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:25 2022] DELETE /identity/v3/projects/b18670978b6145bcb80ac9c9007b1e02 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:25.423115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.423115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.423115 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.442080 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:25.443296 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:25.502656 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1437/2874] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:25.510478 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.510766 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.511042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.530517 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:25.531713 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:25.540026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1438/2875] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:25.544134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.554424 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d8123-6535-41ef-b36b-167982a4ed9b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.558999 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.559597 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.559875 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.560173 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.560811 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:57:25.561112 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.562705 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:25.574139 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1438/2876] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:57:25.580028 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.588165 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a57fb-cfcb-443e-88a0-b3ef62ef8a3e None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.592907 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.593498 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.593772 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.594037 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.594690 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.596674 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:25.675780 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1439/2877] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:25.679945 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.690033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99d1e-12b4-4a9f-b87d-f8caa88733ee admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.747781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.748418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.748700 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.748970 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.749591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.751494 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:25.759361 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:25.880465 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1439/2878] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:25 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:25.884539 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.895336 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d7645-5033-47af-8c35-87f1664e880f admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.900870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.901466 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.901745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.902015 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.902679 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.904312 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:25.915126 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1440/2879] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:25.919234 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.929912 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a6321-4965-4200-875e-ed9118572a49 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.934501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd6cad-c7c2-4b66-ae03-c54de0233bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.935091 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6cad-c7c2-4b66-ae03-c54de0233bfe admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.935421 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6cad-c7c2-4b66-ae03-c54de0233bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.935685 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6cad-c7c2-4b66-ae03-c54de0233bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.936357 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd6cad-c7c2-4b66-ae03-c54de0233bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.937930 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd6cad-c7c2-4b66-ae03-c54de0233bfe admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:25.949171 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1440/2880] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:25.953270 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:25.963924 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc641f94-78ae-43d2-afd1-274818e871f3 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:25.968566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:25.969213 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:25.969492 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:25.969767 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] PATH_INFO: `/v3/projects/dda9b0712c4d47a4a366f7545a83b539/users/11555e2580844a559969dd8dcbb05f24/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:25.989109 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda9b0712c4d47a4a366f7545a83b539, user_id=11555e2580844a559969dd8dcbb05f24, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:25.992411 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.074452 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1441/2881] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:25 2022] PUT /identity/v3/projects/dda9b0712c4d47a4a366f7545a83b539/users/11555e2580844a559969dd8dcbb05f24/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:26.084914 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.085200 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.085506 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.113742 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] MFA Rules not processed for user `11555e2580844a559969dd8dcbb05f24`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:26.114956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:26.179259 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1441/2882] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:26.183801 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.211487 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:26.247260 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df966c2f-8a8c-49a7-b198-87e76fb425ff admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.252128 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.252888 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.253168 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.253446 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] PATH_INFO: `/v3/projects` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.254061 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.256045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.321459 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1442/2883] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:26 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:26.325725 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.337036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68761b6-7074-422a-81e9-1666a16764a7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.392009 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.392649 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.392931 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.393207 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.393834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.395748 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.474674 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1442/2884] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:26 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:26.478549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.488996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce5d02-25ba-4f13-8575-d92af4e1112e admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.494383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.494978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.495259 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.495530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.496242 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.497843 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.509231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1443/2885] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:26.513342 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.523347 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbe299-904c-416b-8ab3-8c44497449f8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.528137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.528801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.529074 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.529344 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] PATH_INFO: `/v3/projects/af4e99c094c84cb9994624c6c51535ac/users/693f3e514f8a4705bcd4e70d26868456/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.544082 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4e99c094c84cb9994624c6c51535ac, user_id=693f3e514f8a4705bcd4e70d26868456, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.546842 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.635431 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1443/2886] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:26 2022] PUT /identity/v3/projects/af4e99c094c84cb9994624c6c51535ac/users/693f3e514f8a4705bcd4e70d26868456/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:26.639532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.649871 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56609a-8695-449c-ae41-fdc06ed09d73 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.704472 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58938cd6-eb71-400b-b27d-56ef3b7d1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.705057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938cd6-eb71-400b-b27d-56ef3b7d1348 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.705338 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938cd6-eb71-400b-b27d-56ef3b7d1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.705621 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938cd6-eb71-400b-b27d-56ef3b7d1348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.706222 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58938cd6-eb71-400b-b27d-56ef3b7d1348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.707814 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58938cd6-eb71-400b-b27d-56ef3b7d1348 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.717748 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1444/2887] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:26.721876 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.732706 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e48b6-95a4-48a1-a9a8-e2bff5072b3d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.737266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.737915 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.738194 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.738466 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] PATH_INFO: `/v3/projects/af4e99c094c84cb9994624c6c51535ac/users/693f3e514f8a4705bcd4e70d26868456/roles/c63cf2d08b334ece887cbd2702835968` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.740438 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4e99c094c84cb9994624c6c51535ac, user_id=693f3e514f8a4705bcd4e70d26868456, role_id=c63cf2d08b334ece887cbd2702835968)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.743684 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:26.753806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1444/2888] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:26 2022] PUT /identity/v3/projects/af4e99c094c84cb9994624c6c51535ac/users/693f3e514f8a4705bcd4e70d26868456/roles/c63cf2d08b334ece887cbd2702835968 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:26.765675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.765966 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.766244 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.794074 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] MFA Rules not processed for user `693f3e514f8a4705bcd4e70d26868456`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:26.795283 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:26.861204 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1445/2889] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:26.872066 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:26.873834 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:26.896719 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:26.932318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bd44-a9cc-47fe-8c66-f5e597655c0d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:26.936000 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d99269-1cb7-45a2-8ff1-877f62b7b599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:26.936660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d99269-1cb7-45a2-8ff1-877f62b7b599 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:26.936969 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d99269-1cb7-45a2-8ff1-877f62b7b599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:26.937239 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d99269-1cb7-45a2-8ff1-877f62b7b599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:26.993617 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d99269-1cb7-45a2-8ff1-877f62b7b599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:26.994977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d99269-1cb7-45a2-8ff1-877f62b7b599 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:27.003776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1445/2890] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:30.081361 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:30.103041 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:30.143519 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34be8dd-8882-4801-b9e9-67154f7d5de9 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:30.148680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:30.149565 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:30.149851 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:30.150230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] PATH_INFO: `/v3/users/11555e2580844a559969dd8dcbb05f24` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:30.151346 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11555e2580844a559969dd8dcbb05f24)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:30.153304 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:30.342830 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11555e2580844a559969dd8dcbb05f24'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:30.403855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11555e2580844a559969dd8dcbb05f24'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:30.411392 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11555e2580844a559969dd8dcbb05f24'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:30.412026 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11555e2580844a559969dd8dcbb05f24'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:30.413239 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11555e2580844a559969dd8dcbb05f24'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:30.431403 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1446/2891] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:30 2022] DELETE /identity/v3/users/11555e2580844a559969dd8dcbb05f24 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:30.436051 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:30.436581 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:30.436784 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:30.456358 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:30.457557 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:30.517223 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1446/2892] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:30.527340 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:30.528751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:30.584104 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3d7bf-67dd-46d0-811a-7bb8a29ccd37 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:30.587692 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:30.588310 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:30.588583 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:30.588847 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:30.597534 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:30.598896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:30.608549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1447/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:31.007798 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:31.009388 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:31.021267 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ca290-0e8a-4997-8bb4-63f4ccaf1445 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:31.026676 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:31.027980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:31.028320 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:31.028595 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] PATH_INFO: `/v3/projects/dda9b0712c4d47a4a366f7545a83b539` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:31.029902 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda9b0712c4d47a4a366f7545a83b539)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:31.031806 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:31.167033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda9b0712c4d47a4a366f7545a83b539', 'request_id': 'req-e272c63f-0794-4581-bc68-7a031dbcbeb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.237666 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda9b0712c4d47a4a366f7545a83b539', 'request_id': 'req-e272c63f-0794-4581-bc68-7a031dbcbeb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.243678 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda9b0712c4d47a4a366f7545a83b539', 'request_id': 'req-e272c63f-0794-4581-bc68-7a031dbcbeb4'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.250776 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:31.255318 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] The token cache is being invalidate because project dda9b0712c4d47a4a366f7545a83b539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:31.255603 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.256283 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.258264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1447/2894] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:31 2022] DELETE /identity/v3/projects/dda9b0712c4d47a4a366f7545a83b539 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:31.262327 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:31.272745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82e88c-cd20-4625-b57b-861aff4b78a4 service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:31.312566 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:31.329038 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:31.329694 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:31.329978 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:31.330253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] PATH_INFO: `/v3/users/693f3e514f8a4705bcd4e70d26868456` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:31.331368 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=693f3e514f8a4705bcd4e70d26868456)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:31.333370 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:31.554619 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '693f3e514f8a4705bcd4e70d26868456'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.651305 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '693f3e514f8a4705bcd4e70d26868456'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.658512 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '693f3e514f8a4705bcd4e70d26868456'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.659140 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '693f3e514f8a4705bcd4e70d26868456'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.659743 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-659f79fa-ec2f-4fc1-a1ad-dcfed76a4ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '693f3e514f8a4705bcd4e70d26868456'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:31.676959 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1448/2895] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:31 2022] DELETE /identity/v3/users/693f3e514f8a4705bcd4e70d26868456 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:32.093199 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.094608 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:32.107928 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c63f-0794-4581-bc68-7a031dbcbeb4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.166979 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.167610 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.167888 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.168187 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] PATH_INFO: `/v3/projects/af4e99c094c84cb9994624c6c51535ac` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.169297 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4e99c094c84cb9994624c6c51535ac)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.171193 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.269375 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4e99c094c84cb9994624c6c51535ac', 'request_id': 'req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:32.325619 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4e99c094c84cb9994624c6c51535ac', 'request_id': 'req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:32.331387 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4e99c094c84cb9994624c6c51535ac', 'request_id': 'req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:32.343118 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] The token cache is being invalidate because project af4e99c094c84cb9994624c6c51535ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:32.343498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:32.344163 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b8825d00-d7ca-48d1-83df-fbd8f38fc72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:32.345775 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1448/2896] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:32 2022] DELETE /identity/v3/projects/af4e99c094c84cb9994624c6c51535ac => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:32.351673 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.352199 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.352323 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.371413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:32.372643 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:32.432815 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1449/2897] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:32.440802 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.441088 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.441501 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.462638 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:32.464134 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:32.472116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1449/2898] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 14:57:32.476400 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.486070 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96496-c11d-4d41-acaf-c85356bb52a7 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.490620 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.491751 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.492023 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.492321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.492956 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:57:32.493253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.494838 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.506012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1450/2899] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:57:32.511579 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.520885 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb00e5-e65b-4dcd-8059-e9f02aeac4ef None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.526022 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.526626 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.526904 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.527176 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.527869 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.529801 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.604479 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1450/2900] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:32 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:32.608595 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.619139 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47d51-1279-42ad-91a6-3f3d77bb39ad admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.664009 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:32.675623 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.676249 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.676532 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.676805 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.677424 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.679969 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.748824 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1451/2901] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:32 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:32.752845 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.763180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defdd69-c846-437b-89c8-89307dd0c15e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.767791 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.768498 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.768778 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.769056 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.769696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.771293 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.781747 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1451/2902] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:32.785675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.796383 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b56939-159c-469c-8f0b-76a4dd3f7045 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.801442 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9041114-87d9-4819-9c07-72a7d27a859c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.802033 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9041114-87d9-4819-9c07-72a7d27a859c admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.802315 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9041114-87d9-4819-9c07-72a7d27a859c admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.802582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9041114-87d9-4819-9c07-72a7d27a859c admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.803855 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9041114-87d9-4819-9c07-72a7d27a859c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.805582 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9041114-87d9-4819-9c07-72a7d27a859c admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.816267 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1452/2903] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 14:57:32.820278 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:32.830443 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8975-5149-4d21-8c7b-00348993015e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:32.835319 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:32.835968 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.836281 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.836558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] PATH_INFO: `/v3/projects/cba5c5444c914a74a4154c898424a435/users/6e6ad2a1d0704b51bc0edd0984d5b64d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.851758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba5c5444c914a74a4154c898424a435, user_id=6e6ad2a1d0704b51bc0edd0984d5b64d, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:32.856995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:32.945994 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1452/2904] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:32 2022] PUT /identity/v3/projects/cba5c5444c914a74a4154c898424a435/users/6e6ad2a1d0704b51bc0edd0984d5b64d/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 14:57:32.964413 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:32.964702 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:32.964975 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:32.993489 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] MFA Rules not processed for user `6e6ad2a1d0704b51bc0edd0984d5b64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:32.994695 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:33.057190 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1453/2905] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:33.068418 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:33.070159 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:33.098190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:33.126956 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f5d74-9a68-4a97-955b-7a65bb323b47 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:33.131120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fbc5e-34b2-441e-9201-56dad705d969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:33.131744 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fbc5e-34b2-441e-9201-56dad705d969 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:33.132021 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fbc5e-34b2-441e-9201-56dad705d969 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:33.132341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fbc5e-34b2-441e-9201-56dad705d969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:33.140473 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fbc5e-34b2-441e-9201-56dad705d969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:33.141916 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fbc5e-34b2-441e-9201-56dad705d969 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:33.151497 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1453/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:36.301251 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:36.362541 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f99fc-9e6d-474c-b359-0abf664bc4a6 None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:36.367594 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:36.368246 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:36.368530 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:36.368802 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] PATH_INFO: `/v3/users/6e6ad2a1d0704b51bc0edd0984d5b64d` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:36.369909 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6ad2a1d0704b51bc0edd0984d5b64d)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:36.371868 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:36.545005 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6ad2a1d0704b51bc0edd0984d5b64d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:36.617683 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6ad2a1d0704b51bc0edd0984d5b64d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:36.622422 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:36.625382 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6ad2a1d0704b51bc0edd0984d5b64d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:36.626004 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6ad2a1d0704b51bc0edd0984d5b64d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:36.626681 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6ad2a1d0704b51bc0edd0984d5b64d'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:36.643689 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1454/2907] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:36 2022] DELETE /identity/v3/users/6e6ad2a1d0704b51bc0edd0984d5b64d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:36.648165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:36.648696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:36.648696 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:36.668838 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:36.670090 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:36.728137 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1454/2908] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:36.738308 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:36.739700 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:36.795996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bd808-b575-4fcd-b804-88dd485c1d9a admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:36.799553 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae033b-bb2c-45e3-90c1-86bc69248daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:36.800164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae033b-bb2c-45e3-90c1-86bc69248daa service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:36.800447 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae033b-bb2c-45e3-90c1-86bc69248daa service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:36.800718 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae033b-bb2c-45e3-90c1-86bc69248daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:36.812483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae033b-bb2c-45e3-90c1-86bc69248daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:36.813936 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae033b-bb2c-45e3-90c1-86bc69248daa service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:36.823944 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1455/2909] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:37.333406 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:37.334827 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:37.346937 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc27a4-1d82-46a7-a604-dfdc82a941f3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:37.354005 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:37.355707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:37.355707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:37.355707 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] PATH_INFO: `/v3/projects/cba5c5444c914a74a4154c898424a435` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:37.357277 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba5c5444c914a74a4154c898424a435)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:37.359980 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:37.460877 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba5c5444c914a74a4154c898424a435', 'request_id': 'req-dff89c94-4684-461c-bfdc-750a54b34070'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:37.523751 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba5c5444c914a74a4154c898424a435', 'request_id': 'req-dff89c94-4684-461c-bfdc-750a54b34070'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:37.530103 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba5c5444c914a74a4154c898424a435', 'request_id': 'req-dff89c94-4684-461c-bfdc-750a54b34070'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:37.541763 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] The token cache is being invalidate because project cba5c5444c914a74a4154c898424a435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:37.542054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:37.542677 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dff89c94-4684-461c-bfdc-750a54b34070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:37.544180 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1455/2910] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:37 2022] DELETE /identity/v3/projects/cba5c5444c914a74a4154c898424a435 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:37.549230 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:37.549740 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:37.549853 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:37.569483 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:37.570680 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:37.628280 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1456/2911] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:37.637410 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:37.637698 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:37.637972 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:37.658266 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:37.659469 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:37.667977 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1456/2912] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:37.672396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:37.682024 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485935f-5587-4101-abc0-cd0d24f960be None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:37.686625 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:37.687224 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:37.687503 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:37.687781 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] PATH_INFO: `/v3/domains` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:37.688470 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81804) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 14:57:37.688775 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:37.690389 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:37.702117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1457/2913] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Sep 7 14:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 14:57:37.706615 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:37.718036 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f029885-3ab7-45a4-a4c7-b14fe7f065ab None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:37.724660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:37.725341 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:37.726251 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:37.726540 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:37.727190 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:37.729116 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:37.810033 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1457/2914] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Sep 7 14:57:37 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:37.814231 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:37.827046 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2935e9-e935-4bd4-8ade-7bdd333e1913 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:37.883302 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:37.883831 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:37.884172 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:37.884451 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:37.885077 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:37.886996 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:37.898549 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:38.035164 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1458/2915] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Sep 7 14:57:37 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 14:57:38.040441 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:38.050639 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4a89a-efec-41e5-99b7-ab9f1a55e4e3 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:38.055170 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:38.056794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.056794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.056794 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.058241 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:38.058545 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:38.068487 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1458/2916] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:38.073179 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:38.083233 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c83dee-cdea-48f6-8257-fa1452ef00b5 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:38.088527 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a5e04-3253-4675-a63a-3a6050e2ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:38.089116 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a5e04-3253-4675-a63a-3a6050e2ad3b admin admin] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.089396 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a5e04-3253-4675-a63a-3a6050e2ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.089669 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a5e04-3253-4675-a63a-3a6050e2ad3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.090300 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a5e04-3253-4675-a63a-3a6050e2ad3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:38.091896 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a5e04-3253-4675-a63a-3a6050e2ad3b admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:38.102248 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1459/2917] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Sep 7 14:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 14:57:38.106258 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:38.116057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8c8b-f036-4017-a604-cd04d433d63c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:38.120588 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:38.121228 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.121508 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.121781 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] PATH_INFO: `/v3/projects/9e0d9b9d58e146f28ceb660455742735/users/254cc70d739f4eaa859f21f53aae370c/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.129995 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:38.137044 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0d9b9d58e146f28ceb660455742735, user_id=254cc70d739f4eaa859f21f53aae370c, role_id=6b9e1bb42d5a4e288cf4a4cfa134ee74)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:38.139660 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:38.212087 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1459/2918] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Sep 7 14:57:38 2022] PUT /identity/v3/projects/9e0d9b9d58e146f28ceb660455742735/users/254cc70d739f4eaa859f21f53aae370c/roles/6b9e1bb42d5a4e288cf4a4cfa134ee74 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:38.222670 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] REQUEST_METHOD: `POST` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.222955 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.223228 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.250942 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.auth.core [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] MFA Rules not processed for user `254cc70d739f4eaa859f21f53aae370c`. Rule list: `[]` (Enabled: `True`). {{(pid=81804) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:38.252347 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:38.315195 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1460/2919] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:38.326788 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:38.328525 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:38.386299 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559535e-d772-49f3-b29e-7d81bbb920ad admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:38.390502 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcc93d-ee72-4a96-b3d5-69ff919ecb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83045f9a43b48d7bbcce339f7bfa5a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:38.391120 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcc93d-ee72-4a96-b3d5-69ff919ecb92 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.391447 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcc93d-ee72-4a96-b3d5-69ff919ecb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.391765 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcc93d-ee72-4a96-b3d5-69ff919ecb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.400591 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcc93d-ee72-4a96-b3d5-69ff919ecb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:38.402027 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcc93d-ee72-4a96-b3d5-69ff919ecb92 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:38.410727 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1460/2920] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:57:38.435675 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:38.499198 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118558c-737a-4593-9437-4f19a504a6ee None None] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:38.503696 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:38.504351 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.504632 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.504899 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] PATH_INFO: `/v3/users/254cc70d739f4eaa859f21f53aae370c` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.506011 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254cc70d739f4eaa859f21f53aae370c)` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:38.508635 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:38.686984 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254cc70d739f4eaa859f21f53aae370c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:38.782870 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254cc70d739f4eaa859f21f53aae370c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:38.790610 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254cc70d739f4eaa859f21f53aae370c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:38.791253 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254cc70d739f4eaa859f21f53aae370c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:38.798045 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.notifications [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254cc70d739f4eaa859f21f53aae370c'} {{(pid=81804) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:38.801954 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:38.810117 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1461/2921] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Sep 7 14:57:38 2022] DELETE /identity/v3/users/254cc70d739f4eaa859f21f53aae370c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:38.814645 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.815165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.815165 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.834264 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] MFA Rules not processed for user `f0b504610bde4460815c46c6c6298f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 14:57:38.835533 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:38.893709 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1461/2922] 10.222.0.57 () {58 vars in 958 bytes} [Wed Sep 7 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 14:57:38.903827 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:38.905348 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:38.962441 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90ab7f-2143-46c2-a21e-a17664652b48 admin admin] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:38.965982 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b50c8a35e28e44bba3a4b4e56057a3fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:38.966587 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:38.966866 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:38.967223 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:38.977376 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:38.978832 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:38.988408 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1462/2923] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 14:57:39.407524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:57:39.407524 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:57:39.421568 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926c30f-88b4-40fa-a7a3-54b8eab9285f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:57:39.426400 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0b504610bde4460815c46c6c6298f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c46f28c367444249b347e64385a501e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:57:39.427758 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:57:39.428057 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:57:39.428369 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] PATH_INFO: `/v3/projects/9e0d9b9d58e146f28ceb660455742735` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:57:39.429578 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0d9b9d58e146f28ceb660455742735)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:57:39.431558 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:57:39.536054 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0d9b9d58e146f28ceb660455742735', 'request_id': 'req-cedbe923-441a-4d16-a7f3-35f6f01aa055'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:39.582636 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0d9b9d58e146f28ceb660455742735', 'request_id': 'req-cedbe923-441a-4d16-a7f3-35f6f01aa055'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:39.589257 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0d9b9d58e146f28ceb660455742735', 'request_id': 'req-cedbe923-441a-4d16-a7f3-35f6f01aa055'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:39.600548 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] The token cache is being invalidate because project 9e0d9b9d58e146f28ceb660455742735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 14:57:39.600836 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:39.601470 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cedbe923-441a-4d16-a7f3-35f6f01aa055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 14:57:39.602958 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1462/2924] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Sep 7 14:57:39 2022] DELETE /identity/v3/projects/9e0d9b9d58e146f28ceb660455742735 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 14:57:48.984079 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:57:49.597274 dv-cinder-801354-8-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 14:58:53.274745 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] Authenticating user token {{(pid=81804) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 14:58:53.276677 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:58:53.345329 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df956591-8d48-461f-b50f-c75a323eb60f service neutron] Validating token access rules against request {{(pid=81804) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 14:58:53.404152 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53d54b01ab3c4003b5f13e20c0849db1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0f1bc97dae24914a603354fcc9a8d41', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81804) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 14:58:53.404783 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] REQUEST_METHOD: `GET` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 14:58:53.405057 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] SCRIPT_NAME: `/identity` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 14:58:53.405321 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81804) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 14:58:53.407042 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.fernet_utils [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81804) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 14:58:53.418012 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 14:58:53.419602 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9816d-672c-4474-9fd5-5f8b1fefcffe service placement] RBAC: Authorization granted {{(pid=81804) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 14:58:53.429418 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: [pid: 81804|app: 0|req: 1463/2925] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Sep 7 14:58:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 14:59:03.425186 dv-cinder-801354-8-iscsi devstack@keystone.service[81804]: DEBUG dbcounter [-] [81804] Writing DB stats keystone:SELECT=58 {{(pid=81804) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}